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MEMORANDUM  FOR  DIRECTOR,  DEFENSE  LOGISTICS  AGENCY 


SUBJECT:  Report  on  the  Defense  Contract  Management  Command  Management  of 
Quality  Assurance  Resources  ^Report  No.  95-166) 


We  are  providing  this  report  for  your  review  and  comments.  Comments  on  a 
draft  of  this  report  were  considered  in  preparing  the  final  report. 

DoD  Directive  7650.3  requires  that  all  recommendations  be  resolved  promptly. 
We  request  that  the  Defense  Logistics  Agency  provide  additional  comments  on  the 
revised  recommendations  by  June  12,  1995.  See  the  table  at  the  end  of  each  finding 
for  the  specific  requirements  for  comments. 

Questions  on  this  audit  should  be  directed  to  Mr.  Salvatore  D.  Guli,  Audit 
Program  Director,  at  (703)  604-9500  (DSN  664-9500)  or  Mr.  C.  J.  Richardson,  Audit 
Project  Manager,  at  (703)  604-9582  (DSN  664-9582).  Copies  of  this  report  will  be 
distributed  to  the  organizations  listed  in  Appendix  H.  The  audit  team  members  are 
listed  inside  the  back  cover. 

David  K.  Steensma 
Deputy  Assistant  Inspector  General 
for  Auditing 
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Defense  Contract  Management  Command  Management  of 
Quality  .^surance  Resources 


Executive  Summary 


Introduction.  The  Defense  Contract  Management  Command  (the  Command) 
personnel  involved  in  quality  assurance  oversight  represent  nearly  35  percent  of  die 
Command's  contract  administration  worldforce.  As  of  October  1994,  the  Command 
employed  5,567  quality  assurance  specialists,  at  a  cost  of  $295.2  million.  Since  1991, 
the  Command  has  reduced  the  number  of  quality  assurance  specialists  about  22  percent 
from  the  1991  total  of  7,125.  Since  1990,  the  Command  has  attempted  to  convert  to 
process-oriented  quali^  assurance  through  the  In-Plant  Quality  Ev^uation  program, 
and  more  recently  in  1994,  by  implementing  the  Process  Oriented  Contract 
Administration  Services  program.  Process  Orient^  Contract  Administration  Services 
methodology  and  teaming  concepts  are  not  consistent  with,  and  present  problems  for, 
the  traditional  methods  the  Command  uses  to  determine  personnel  levels.  Resourcing 
quality  assurance  personnel  needs  is  a  problem  made  more  difficult  by  the  cultural 
organizational  changes  presented  by  the  new  program. 

Objectives.  Our  prim^  audit  objectives  were  to  evaluate  the  Command's  policies  and 
procedures  for  managing  quality  assurance  resources  and  the  bases  for  determining 
appropriate  personnel  levds.  Additionally,  we  were  to  evaluate  compliance  with 
previous  audit  recoinmendations  and  applicable  internal  controls.  We  eliminated  the 
announced  audit  objective  to  evaluate  the  success  of  programs  for  training  quality 
assurance  personnel  because  of  time  constraints. 

Audit  Results.  The  Command  did  not  efrectively  manage  quality  assurance  specialists 
to  adequately  implement  the  bi-Plant  Quality  Evaluation  program.  After  4  years, 
quality  assurance  specialists  performed  detail  examinations  of  only  5  percent  of  the 
1,818  manufacturing  processes  identified  at  13  audit  sites.  The  audit  did  not  attempt  to 
identify  nonconforming  products  from  the  13  audit  sites.  However,  the  Command 
lacked  the  evaluation  and  supervisory  processes  needed  to  establish  accountability  for 
implementing  quality  assurance  actions  under  flie  In-Plant  Quality  Evaluation  and  the 
Process  (Ment^  Contract  Administration  Services  programs.  Further,  the  Command 
cannot  ensure  that  the  products  accq)ted  were  product  under  reliable  processes  that 
would  consistently  produce  a  conforming  product  without  detailed  examination  of  die 
manufacturing  processes  (Finding  A). 

The  Command  did  not  establish  an  effective  method  to  determine  the  number  of  quality 
assurance  personnel  needed  to  accomplish  the  contract  quality  assurance  program.  As  a 
result,  the  Command  did  not  have  an  adequate  basis  for  estimating  the  staftog  required 
for  quality  assurance  in  a  process-oriented  quality  assurance  environment  (Finding  B). 


Internal  Controls.  Internal  controls  were  not  adequate  to  hold  Command  management 
and  staff  accountable  for  effective  implementation  of  the  In-Plant  Quality  Evaluation 
program.  Also,  internal  controls  were  not  adequate  for  documenting  workload 
activities  as  a  basis  for  staffing  levels  to  provide  adequate  quality  assurance  surveillance 
of  DoD  contracts.  We  consider  those  internal  control  weaknesses  to  be  material.  In 
addition,  the  Command  had  an  insufficient  basis  for  rqwrting  that  quality  assurance 
was  a  low-risk  area  under  the  DoD  Internal  Management  Control  Program.  See  Part  I 
for  the  internal  controls  reviewed  and  Part  n  for  details  of  the  weaknesses. 

Potential  Benefits  of  Audit.  Implementation  of  the  recommendations  will  result  in 
better  management  control  and  more  efficient  use  of  quality  assurance  resources.  We 
are  unable  to  quantify  the  monetary  benefits  that  could  be  realized  by  improving 
controls  because  the  benefit  of  better  quality  weapons  and  equipment  is  undeterminable. 
Appendix  F  lists  the  potential  benefits  resulting  from  the  audit; 

Sununaiy  of  Reconunendations.  We  recommend  that  the  Defense  Logistics  Agency 
(DLA)  establish  a  system  of  accoimtability  and  measurement  over  implementation  of 
process-oriented  quality  assui^ce.  We  further  recommend  that  DLA  establish  standard 
performance  plans  for  quality  assurance  specialists  and  supervisors  to  hold  them 
responsible  for  fully  implementing  specific  process-oriented  quality  assurance 
functions.  In  addition,  we  recommend  that  quality  assurance  specialists  document  the 
estimated  work  required  to  perform  process-orient^  quality  assurance  on  each  contract 
assigned,  and  that  the  Command  districts  use  those  work  estimates  as  the  basis  for 
future  budget  requests. 

Management  Comments.  DLA  concurred  with  recommendations  to  define  critical 
manufactuimg  processes  Md  to  develop  quality  assurance  manpower  estimates  and 
corresponding  budget  estimates  based  on  documented  quality  assurance  workload. 
DLA  did  not  concur  with  the  recommendations  to  establish  a  system  of  accountability 
for  quality  assurance  specialists  and  their  supervisors  to  fully  implement  process 
oriented  quabty  assurance.  Also,  DLA  did  not  concur  with  recommendations  designed 
to  measure  performance  or  to  base  estimated  workload  on  actual  quality  assurance 
tasks. 

Audit  Response.  We  consider  the  DLA  comments  to  be  partially  responsive. 
However,  we  believe  that  if  DLA  establishes  new  quality  assurance  programs,  it  should 
hold  quality  assurance  specialists  and  their  supervisors  accountable  for  implementing 
the  programs.  DLA  needs  to  measure  the  implementation  and  the  results  of  the  new 
process-oriented  quaUty  assurance  programs.  As  a  result  of  DLA  comments,  we 
revi^  seven  recommendations.  We  deleted  specific  worldoad-estimating  reporting 
requirements  and  added  a  requirement  to  establish  a  system  for  informing  the 
Commander,  Defense  Contract  Management  Command,  of  the  specific  contracts  that 
quality  assurance  specialists  either  would  not  be  able  to  or  were  not  able  to  support 
with  an  adequate  level  of  quality  assurance.  We  request  that  DLA  provide  additional 
comments  to  the  final  report  by  June  12,  1995. 
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Part  I  -  Introduction 


Introduction 


Background 


The  Federal  Acquisition  Regulation  (FAR)  part 46,  "Quality  Assurance," 
provides  policies  to  ensure  that  supplies  acquired  through  Government  contracts 
conform  to  the  contract's  quality  requirements.  The  responsibility  for  quality  is 
divided  between  the  contractor  and  the  Government.  The  contractor  is 
responsible  for  controlling  the  quality  of  supplies  during  production  and  offering 
to  the  Government  for  acceptance  only  those  supplies  that  conform  to  contract 
requirements.  Government  agencies  are  responsible  for  ensuring  that  supplies 
offered  by  contractors  meet  contract  requirements. 

Quality  Assurance  Management  in  DoD.  The  Defense  Contract  Management 
Command  (DCMC),  a  subordinate  command  of  the  Defense  Logistics  Agency 
(DLA),  provides  contract  quality  assurance  oversight  as  part  of  the  contract 
administration  function  for  DoD  contracts.  Organizationally,  DCMC  manages 
contract  administration  in  three  Defense  contract  management  districts 
(DCMDs)  to  support  DoD  organizations  in  the  continental  United  States. 
Contract  administration  is  principally  conducted  at  Defense  contract 
management  area  operations  (DCMAOs)  for  a  number  of  medium-to-small  size 
contractors  and  at  Defense  plant  representative  offices  (DPROs)  for  the  largest 
Defense  contractors. 

A  major  portion  of  the  contract  administration  function  is  related  to  quality 
assurance.  DCMC  provides  quality  assurance  for  supplies  accepted  at  the 
source  (the  contractor's  plant)  through  the  oversight  efforts  of  quality  assurance 
specialists.  Quality  Assurance  Specialist  are  organizationally  assigned  to  DPRO 
residencies  at  the  site  of  large  contractors,  to  the  DCMAO  residencies  at  the 
sites  of  smaller  contractors,  or  as  non-resident  Quality  Assurance  Specialist  for 
contractors  who  have  only  a  few  DoD  contracts  requiring  acceptance  at  the 
source  of  supply. 

Quality  Assurance  through  Defect  Detection.  Before  1990,  Quality 
Assurance  Specialist  performed  in-plant  quality  assurance  functions,  largely  by 
detecting  and  rejecting  nonconformances  produced  by  contractors.  Primary 
policy  was  contained  in  Defense  Logistic  Agency  Manual  8200.1,  "Defense  In- 
Plant  Quality  Assurance,"  August  30,  1976.  Quality  Assurance  Specialist 
concentrated  on  using  procedural  reviews  and  audits  and  product  inspection  to 
detect  and  avoid  the  acceptance  of  nonconforming  material.  Before  1990,  each 
Military  Departments'  plant  representative  offices  performed  quality  assurance 
in  a  manner  unique  to  the  Militaty  Department  and  in  accordance  with  Military 
Department  regulations  and  policies. 

Consolidation  of  Contract  Administrative  Services.  In  July  1989,  the 
Secretary  of  Defense  consolidated  all  contract  administrative  services  under  the 
direction  of  DCMC.  Following  the  consolidation  of  the  Military  Department's 
plant  representative  offices  and  Defense  contract  administration  services 
regions,  DLA  initiated  a  process-oriented  approach  to  quality  assurance  through 
the  In-Plant  Quality  Evaluation  program  (IQUE).  In  1992,  DCMC  developed 
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the  concept  of  including  IQUE  in  all  parts  of  contract  administration  through  the 
Process  Oriented  Contract  Administration  Services  program  (PROCAS). 
PROCAS  was  initiated  in  all  of  the  DPROs  by  1994.  Under  IQXJE,  DCMC 
primarily  relies  on  the  contractor's  quality  control  system  for  ensuring  that 
supplies  meet  contract  requirements. 

Quality  Assurance  Resources.  As  of  October  1994,  DCMC  employed 
6,829  people  assigned  to  the  three  continental  United  States  DCMDs  to  perform 
the  quality  assurance  mission.  Of  the  6,829  people  assigned,  5,567  were 
quality  assurance  specialist.  In  FY 1993,  quaHty  assurance  specialists 
performed  quality  assurance  functions  at  contractor  sites  for  190,383  contracts, 
valued  at  $103.5  billion.  The  cost  of  the  6,829  quality  assurance  personnel  was 
$415  million. 

Glossary  of  Terms.  Appendix  A  describes  the  terms  associated  with  process- 
oriented  quality  assurance  such  as  process,  critical  process,  process  proofing, 
reproofing  processes,  product  audits,  product  quality  deficiency  investigations 
and  nonconforming  materials  review. 


Objectives 


The  primary  audit  objectives  were  to  evaluate  DCMC  policies  and  procedures 
for  managing  quality  assurance  resources  and  the  bases  for  determining 
appropriate  personnel  levels.  Additionally,  we  evaluated  compliance  wiA 
previous  audit  recommendations  and  applicable  internal  controls.  We 
eliminated  the  announced  objective  to  evaluate  the  success  of  programs  for 
training  quality  assurance  personnel  because  of  time  constraints. 


Scope  and  Methodology 


We  reviewed  DCMC  implementation  of  process-oriented  quality  assurance 
applied  to  contracts  closed  during  FYs  1993  and  1994.  We  reviewed  the 
documentation  supporting  the  quality  assurance  process  evaluations  for 
27  programs  at  9  qxxality  assurance  residencies  within  6  DCMAOs  and 
4  DPROs,  within  the  3  DCMDs:  Northeast,  West,  and  South  (Appendix  B). 

We  reviewed  the  workload  documentation  methods  of  quality  assurance 
requirements,  as  they  relate  to  staffing  levels,  for  104  contracts,  valued  at 
$10.8  billion,  assign^  to  10  quality  assurance  residencies  in  6  DCMAOs  and 
3  DPROs  (Appendix  C).  Appendix  G  lists  the  organizations  visited  or 
contacted. 
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Source  of  Audit  Information.  We  did  not  rely  on  computer-processed  data  or 
use  statistical  sampling  procedures  to  achieve  the  audit  objectives.  We  obtained 
our  audit  information  primarily  from  examination  of  DCMC  policies, 
procedures,  and  management  reports;  from  review  of  contract  quality  assurance 
requirements;  from  analysis  of  contractor  processes  identified  and  proofed  since 
1989;  and  from  interviews  with  quality  assurance  personnel  throughout  DCMC. 

Audit  Period  and  Standards.  This  program  audit  was  conducted  from 
September  13,  1993,  through  August  19,  1994,  in  accordance  with  auditing 
standards  issued  by  the  Comptroller  General  of  the  United  States  as 
implemented  by  the  Inspector  General,  DoD.  Accordingly,  the  audit  included 
tests  of  internal  controls  considered  necessary. 


Internal  Controls 


Internal  Controls  Reviewed.  We  evaluated  the  internal  controls  for  assuring 
that  DCMC  appropriately  managed  quality  assurance  resources  and  used 
definite,  measurable  methods  for  determining  appropriate  quality  assurance 
staffing  levels.  We  evaluated  the  internal  controls  for  the  implementation  of 
IQUE  and  PROCAS  as  they  apply  to  quality  assurance  actions. 

From  1989  to  1992,  DLA  reported  nonconforming  material  in  the  DoD  Supply 
System  as  a  material  weakness  as  part  of  the  Federal  Managers'  Financial 
Integrity  Act  Annual  Statement  of  Assurance  in  accordance  with  DoD 
Directive  5010.38,  "Internal  Management  Control  Program,"  April  14,  1987. 
One  of  the  corrective  actions  was  to  implement  IQUE.  The  DLA  Internal 
Management  Control  Plan  for  FYs  1993  through  1997  included  instructions  for 
DCMDs  to  perform  process  risk  assessments  for  each  assessable  unit,  including 
IQUE. 

Adequacy  of  Internal  Controls.  We  reviewed  the  DCMC  implementation  of 
the  DoD  Internal  Management  Control  Program.  The  audit  identified  material 
internal  control  weaknesses  as  defined  by  DoD  Directive  5010.38.  DCMC  did 
not  have  effective  internal  controls  for  the  management  of  quality  assurance 
specialists,  the  appropriate  level  of  quality  assurance  resourcing,  and 
accountability  for  implementing  quality  assurance  actions  under  IQUE  and 
PROCAS.  DCMC  lacked  methods  for  evaluating  and  measuring  the 
implementation  of  IQUE  and  PROCAS  as  related  to  quality  assurance  actions. 
The  DCMDs  did  not  adequately  assess  the  success  of  IQUE  and  PROCAS.  The 
assessments  did  not  evaluate  progress  in  the  effort  toward  the  identification  and 
proofing  of  critical  processes. 

The  overall  lack  of  oversight,  responsibility,  and  accountability  for  DCMC 
Quality  Assurance  Specialist'  implementation  of  IQUE  and  PROCAS  and  the 
DCMC  lack  of  a  valid  basis  for  funding  and  determining  the  staffing  needed  for 
quality  assurance  requirements  constituted  material  internal  control  weaknesses. 
Recommendations  A.2,  A. 3.,  A. 4.,  A.5.,  B.2.,  B.3.,  B.4.,  B.6.  and  B.7.  in 
Part  II  of  the  report,  if  implemented,  will  improve  DLA  implementation  of 
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process  oriented  quality  assurance.  We  could  not  determine  the  potential 
monetary  benefits  that  will  result  from  implementing  the  recommendations 
because  the  benefits  will  result  from  future  actions  to  improve  the  DCMC 
management  of  quality  assurance  resources.  Appendix  F  summarizes  the 
potential  benefits  of  the  audit.  A  copy  of  the  report  will  be  provided  to  the 
senior  official  responsible  for  internal  controls  in  DLA. 


Prior  Audits  and  Other  Reviews 


This  report  is  the  seventh  in  a  series  of  reports  published  since  1990  by  the 
Inspector  General,  DoD,  regarding  quality  assurance  in  DoD.  The  three  most 
recent  and  relevant  reports  are  summarized  below,  and  Appendix  D  provides 
summaries  for  the  other  four  published  reports. 

Inspector  General,  DoD,  Report  No.  95-044,  "Statistical  Process  Control  at 
McDonnell  Douglas  Helicopter  Systems,"  November  28,  1994.  The  report 
states  that  McDonnell  Douglas  Helicopter  has  not  folly  implemented  a 
contractually  required  statistical  process  control  system  at  its  Mesa,  Arizona, 
facility.  The  resident  DPRO  did  not  provide  continuous  oversight  over  the 
implementation  of  the  statistical  processing  control  system.  As  a  result,  DoD 
expended  at  least  $4.4  million  in  statistical  process  control  system  funds  on  the 
AH-64  Apache  attack  helicopter  contracts  without  folly  gaining  the  benefits 
derived  from  the  statistical  process  control  system.  The  report  recommended 
that  DCMC  issue  written  internal  control  objectives  and  verification  techniques 
to  substantiate  the  inclusion  and  administration  of  contractually  required 
statistical  process  control  systems.  DLA  concurred  with  the  recommendation 
and  took  action  to  specifically  focus  on  McDonnell  Douglas  Helicopter's 
statistical  process  control  implementation  process. 

Inspector  General,  DoD,  Report  No.  94-INS-12,  "Inspection  of  the  Defense 
Contract  Management  Command",  September  29,  1994.  The  inspection  report 
objective  was  to  assess  the  implementation  of  the  Defense  Management  Review 
Decision  916,  published  in  1989,  to  consolidate  contract  administration  services 
under  DLA.  Although  DLA  had  made  significant  progress  in  achieving  the 
goals  of  Defense  Management  Review  Decision  916,  improvements  were 
needed  in  three  areas:  manpower  management,  program  support,  and  property 
oversight.  DCMC  does  not  have  a  consistent  process  to  review  and  project 
staffing  requirements  based  upon  quantitative  and  qualitative  workload 
measurement  techniques.  The  inspection  report  recommended  that  the 
Commander,  DCMC,  establish  consistent  manpower  requirements  analysis 
methodologies  so  that  comparable  functions  in  various  parts  of  DCMC  are 
subjected  to  the  same  analysis.  DCMC  concurred  with  the  recommendation  and 
noted  it  was  developing  a  workload  forecast  model  to  project  its  workload  and  a 
staffing  assistance  model  to  quantify  the  relationship  between  workload 
indicators  and  staffing  requirements.  The  estimated  completion  date  is 
September  30,  1995. 
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Inspector  General,  DoD,  Report  No.  94-079,  "DoD  Component  Implementing 
Action  Plans  for  Improving  the  Quality  of  Spare  Parts,"  April  12,  1994.  The 
report  states  that  the  initial  action  plans  the  DoD  Components  prepared  in 
1990  were  short-term  measures  designed  to  address  long-standing  problems. 
While  DoD  Components  continued  to  develop  initiatives  to  improve  the  quality 
of  spare  and  repair  parts,  implementing  action  plans  were  not  effectively  used 
for  managing  quality  programs.  The  DoD  Component  action  plans  did  not  * 
include  the  bases  for  holding  management  officials  accountable  for  achieving 
quality  program  results.  As  a  result,  DoD  Components  did  not  have 
performance  measures,  milestones,  and  feedback  mechanisms  to  measure  the 
effectiveness  of  the  quality  program.  The  report  states  that  DCMC  routinely 
describes  improvements  in  its  methods  for  obtaining  improved  quality  from 
contractors.  However,  the  DLA  Action  Plan  does  not  incorporate  plans  for 
measuring  the  progress  of  DoD  contractors  in  achieving  continuous 
improvement.  The  report  recommended  that  DLA  measure  the  extent  to  which 
each  DoD  contractor's  quality  control  processes  are  reviewed  and  validated. 
DLA  nonconcurred  with  the  recommendation,  stating  that  the  recommended 
activity-based  measure  will  not  significantly  contribute  to  improved  quality. 
PROCAS  will  increase  DCMC  effectiveness  in  reducing  nonconformances. 
One  measure  of  effectiveness  related  to  PROCAS  is  to  determine  whether 
DCMC  is  selecting  processes  for  improvement  that  have  the  most  impact  on  the 
quality  of  the  product.  This  process  identification  and  selection  is  done  in  the 
risk-assessment  process  conducted  at  each  contract  administration  office. 
Another  measure  of  effectiveness  is  the  process  performance,  product  quality 
levels,  or  both  at  each  facility. 

The  DLA  response  was  accepted  on  the  basis  that  Inspector  General,  DoD, 
Report  No.  95-166,  "Defense  Contract  Management  Command's  Management 
of  Quality  Assurance  Resources,"  would  evaluate  the  effect  of  PROCAS  and 
process  performance  on  quality  assurance  effectiveness. 


Other  Matters  of  Interest 


Corrective  Actions  Taken.  DCMAO  Clearwater,  Florida,  in  response  to  the 
audit,  initiated  several  actions  to  improve  the  implementation  of  IQUE. 
DCMAO  Clearwater  issued  a  letter  to  all  quality  assurance  personnel  identifying 
the  need  to  improve  the  documentation  supporting  product  audits,  process 
proofings  and  reproofmgs,  and  corrective  action  measures  taken  with  the 
contractor.  The  management  at  DCMAO  Clearwater  emphasized  clear, 
complete,  and  concise  recording  of  all  activities  and  tasked  quality  assurance 
supervisors  to  provide  assistance  and  guidance  to  quality  assurance  personnel  to 
achieve  these  results.  The  DCMAO  Clearwater  also  established  a  format  to 
inform  its  customers  of  plans  to  reproof  processes.  The  DCMAO  Clearwater 
early  response  to  problems  identified  during  the  audit  represented  timely, 
positive  steps  toward  improving  the  overall  management  of  quality  assurance 
resources. 
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Part  II  -  Findings  and  Recommendations 


Finding  A.  Managing  the  Quality 
Assurance  Workforce 

DCMC  did  not  effectively  manage  the  quality  assurance  workforce. 
Quality  assurance  specialists  and  supervisors  at  all  of  the  13  audit  sites 
did  not  effectively  implement  the  In-Plant  Quality  Evaluation  or  the 
Process-Oriented  Contract  Administration  Services  programs  (IQUE  and 
PROCAS).  Specifically,  these  conditions  occurred  because: 

o  DCMC  did  not  adequately  prioritize  the  need  to  identify  critical 
manufacturing  processes  for  IQUE  and  PROCAS; 

o  DCMC  did  not  hold  quality  assurance  specialists  accountable  for 
implementing  either  IQUE  or  PROCAS; 

o  DCMC  did  not  hold  supervisors  responsible  for  reviewing  and 
evaluating  the  identification  of,  and  proofing  of,  processes; 

o  DCMC  did  not  effectively  follow-up  on  a  1992  action  plan  to 
improve  the  IQUE  program;  and 

o  DCMC  did  not  report  realistic  assessments  of  the  risk  associated 
with  IQUE  in  its  Internal  Management  Control  Plan. 

Consequently,  neither  IQUE  nor  PROCAS  provided  adequate  assurance 
that  the  products  the  Government  accepted  were  produced  under  a 
reliable  process  that  would  consistently  produce  a  conforming  product. 
(The  audit  did  not  attempt  to  determine  if  the  suppliers  at  the  13  audit 
sites  produced  nonconforming  products.)  In  addition,  quality  assurance 
specialists  did  not  take  advantage  of  process-oriented  quality  assurance  to 
establish  a  basis  for  substituting  process  proofing  and  product  audits  for 
more  labor-intensive  and  expensive  mandatory  inspections. 


Background 


FAR  part  46  states  that  the  contract  administration  office  shall  develop  and 
apply  efficient  procedures  for  performing  Government  contract  quality 
assurance  actions.  FAR  part  46  also  requires  that  the  contract  administration 
office  maintain  suitable  records  reflecting  the  nature  of  Government  contract 
quality  assurance  actions  and  decisions  regarding  the  acceptability  of  the 
products,  the  processes,  and  the  requirements,  as  well  as  suitable  records  of 
action  taken  to  correct  defects. 

DLA  Quality  Assurance  Guidance.  DLA  Manuals  (DLAM)  8200.5,  "In-Plant 
Quality  Evaluation  (IQUE),"  Septembers,  1992,  and  8000.5,  "One Book," 
part  II,  chapter  2,  "Process  Oriented  Contract  Administration  Services 
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(PROCAS)"  April  3,  1995,  contain  the  day-to-day  policies  and  procedures  for 
DCMC  personnel  responsible  for  administering  Government  contract  quality 
assurance  and  accepting  products  from  suppliers. 

DCMC  provides  quality  assurance  specialists  guidance  in  DLAMs  8200.5  and 
8000.5  on  how  to  proof  the  adequacy  of  a  contractor's  processes.  Quality 
assurance  specialists  review  and  assess  the  process  to  determine  whether  the 
contractor  adequately  employs  the  following  factors:  people,  methods, 
environment,  equipment,  and  materials  to  achieve  desired  outcomes. 
Specifically,  quality  assurance  specialists  analyze  applicable  contractor  quality 
controls  used  to  keep  manufacturing  processes  wiAin  stated  tolerance  limits, 
quality  assurance  specialists  are  supposed  to  prepare  flowcharts  that  describe 
involved  processes  and  document  each  of  the  contractor  process  factors.  If 
significant  changes  occur  to  the  process  factors,  then  the  quality  assurance 
specialist  should  reproof  the  process. 

Continuous  Improvement  Through  IQUE  and  PROCAS.  The  focus  of 
IQUE  is  to  improve  quality  by  continuously  improving  the  contractor 
production  processes  through  thorough  ongoing  review  and  documentation 
instead  of  attempting  to  detect  nonconformances  in  completed  products. 
Proofing  contractor  processes  allows  for  early  identification  of  problems  and 
provides  a  level  of  confidence  for  items  manufactured  at  contractor  plants. 

PROCAS  expands  on  the  IQUE  philosophy  by  applying  IQUE  to  all  contract 
administration  functions  and  by  providing  a  cross-functional  team  approach  to 
contract  administration.  The  cross-functional  team  approach  is  designed  to 
increase  communication  and  cooperation  between  DCMC,  military  customers, 
industry  contractors,  and  the  Defense  Contract  Audit  Agency.  For  those 
contractors  who  decide  not  to  participate  in  PROCAS,  quality  assurance 
specialists  will  implement  PROCAS  and  IQUE  to  the  maximum  extent  possible, 
including  the  use  of  process  proofing. 

DCMC  stated  in  DLAMs  8200.5  and  in  8000.5  that  quality  assurance  specialists 
operating  under  both  IQUE  and  PROCAS  would  identify  contractor-established 
processes  and  assess  the  processes  and  the  resulting  outcome  to  determine 
whether  products  consistently  conform  to  contract  technical  requirements. 


Quality  Assurance  Program  Management 


DCMC  did  not  effectively  manage  its  quality  assurance  workforce  to  ensure 
adequate  implementation  of  IQUE  and  PROCAS.  DCMC  quality  assurance 
specialists  at  all  13  audit  sites  were  not  effectively  using  methods  for  identifying 
and  proofing  contractor  manufacturing  processes.  Quality  assurance  specialists 
responsible  for  quality  assurance  identified  a  total  of  1,818  processes  for  the 
27  programs  reviewed.  The  quality  assurance  specialist  approach  to  process 
identification  was  inconsistent.  Some  quality  assurance  specialists  identified 
detailed  processes  within  general  areas.  For  example,  the  quality  assurance 
specialists  for  3  of  the  27  programs  (Titan  missile.  Delta  rocket  and  Simplex 
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underwater  cable)  identified  a  total  of  875  processes.  Conversely,  the  quality 
assurance  specialists  for  the  remaining  24  programs  identified  a  total  of  only 
1,001  processes.  Some  of  the  quality  assurance  specialists  responsible  for  the 
24  programs  identified  only  the  broad  manufacturing  processes  and  did  not 
attempt  to  identify  more  specific  processes,  even  though  some  of  the  specific 
processes  may  have  been  critical. 

The  quality  assurance  specialists  determined  that  33  percent,  598  of  the  1,818 
processes,  were  critical  processes.  The  weapon  system  programs  included  in 
our  review  should  have  a  high  percentage  of  critical  processes.  Quality 
assurance  specialist  did  not  make  proofing  of  critical  processes  a  priority  either. 
Of  the  598  critical  manufacturing  processes  identified,  quality  assurance 
specialists  only  proofed  85  critical  manufacturing  processes. 

Quality  assurance  specialist  supervisors  did  not  effectively  manage  the  quality 
assurance  specialist  workforce.  Supervisors  did  not  evaluate,  monitor,  and 
track  quality  assurance  specialist  work  to  ensure  that  IQUE  and  PROCAS 
requirements  were  adequately  satisfied.  Further,  staff  assistance  from  DCMC 
and  the  DCMDs  did  not  generate  documented,  independent  assessments  of 
IQUE  implementation,  and  the  DCMC  Internal  Management  Control  Program 
did  not  provide  realistic  assessments  of  the  risk  associated  with  the  quality 
assurance  function. 

Ineffective  management  of  the  quality  assurance  specialist  workforce,  coupled 
with  the  fact  that  quality  assurance  specialists  and  quality  assurance  specialist 
supervisors  were  not  held  accountable  for  identifying  and  proofing  processes, 
provided  less  than  reasonable  assurance  that  contractor  manufacturing  processes 
could  consistently  produce  conforming  products. 


Identifying  and  Proofing  Critical  Manufacturing  Processes 


DCMC  did  not  have  an  effective  system  for  making  sure  that  critical  processes 
were  identified.  In  addition,  when  critical  processes  were  identified,  quality 
assurance  specialists  did  not  place  a  high  priority  on  proofing  the  critical 
processes. 

Identifying  Critical  Manufacturing  Processes.  DCMC  did  not  adequately 
prioritize  the  need  to  identify  critical  manufacturing  processes.  Of 
1,818  manufacturing  processes  related  to  major  acquisition  programs, 
598  processes  (33  percent)  were  identified  as  critical  manufacturing  processes. 
Manufacturing  processes  related  to  fabrication  and  assembly  of  major  system 
end  items  usually  involve  a  high  number  of  critical  processes  based  on  the 
consequences  of  mission  failure.  The  quality  assurance  specialist  and  the 
quality  assurance  specialist  supervisor  should  make  the  identification  of  critical 
processes  a  documented  priority  teamwork  product.  As  evidenced  by  the  low 
percentage  of  critical  processes  identified,  quality  assurance  specialists  at  9  of 
the  13  audit  sites  did  not  make  the  identification  of  critical  processes  a  priority. 
The  quality  assurance  specialists  at  4  of  the  audit  sites  identified  101  of 
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108  (94  percent)  manufacturing  processes  as  critical.  Quality  assurance 
specialists  at  the  9  remaining  sites  identified  only  29  percent  of  manufacturing 
processes  as  critical. 

The  identification  of  critical  manufacturing  processes  was  extremely  varied  and 
reflected  confusion  as  to  what  a  criticad  manufacturing  process  was.  The 
following  are  examples. 

o  Quality  assurance  specialists  at  Aerojet  in  Sacramento,  California, 
determined  that  all  of  the  324  manufacturing  processes  for  the  Titan  missile 
were  critical.  At  the  same  location,  quality  assurance  specialists  for  the  Delta 
rocket  identified  only  20  of  232  processes  as  critical.  We  compared  the  critical 
processes  of  the  Titan  to  the  Delta  manufacturing  processes  and  determined  that 
58  processes  were  similarly  named  and  that  apparently  the  local  criteria  for 
identifying  critical  manufacturing  processes  was  applied  differently  by  the 
quality  assurance  specialists. 

o  Quality  assurance  specialists  for  the  Patriot  missile  guidance  system 
and  ground  units  at  DPRO  Raytheon  in  Andover,  Massachusetts,  determined 
that  40  of  47  manufacturing  processes  were  critical.  However,  the  quality 
assurance  specialists  for  the  Patriot  missile  warhead,  tail,  fins,  and  final 
assembly  at  Martin  Marietta  in  Orlando,  Florida,  did  not  identify  any  of 
19  manufacturing  processes  as  critical.  The  quality  assurance  specialists  at 
Martin  Marietta  were  in  error,  and  the  quality  assurance  specialist  supervisor 
acknowledged  that  at  least  part  of  the  19  manufacturing  processes  related  to  the 
warhead  and  final  assembly  were  critical. 

Proofing  Critical  Manufacturing  Processes.  Quality  assurance  specialists  for 
the  27  programs  identified  a  total  of  598  critical  manufacturing  processes  but 
only  proofed  85  of  the  critical  processes.  We  concluded  that  identifying  a 
process  as  critical  had  little  influence  on  whether  quality  assurance  specialists 
would  prioritize  the  proofing  of  a  critical  process.  Quality  assurance  specialist 
showed  uncertainty  about  what  constituted  a  critical  process  because  fte  term 
critical  process  was  not  defined.  DCMC  had  not  adequately  emphasized  the 
priority  of  identifying  and  proofing  critical  manufacturing  processes.  DLA 
needs  to  emphasize  the  importance  of  critical  manufacturing  processes  and  needs 
to  develop  guidance  that  clearly  defines  a  critical  manufacturing  process. 


Holding  Quality  Assurance  Specialists  Accountable 


DCMC  delegated  the  responsibility  for  implementation  of  IQUE  to  the  quality 
assurance  specialist  working  with  the  contractor.  However,  the  delegation  did 
not  provide  for  a  system  of  accountability  through  the  chain  of  command  and 
did  not  hold  quality  assurance  specialists  accountable  for  performing  quality 
assurance.  DCMC  needs  to  establish  a  system  of  accountability  that  holds 
quality  assurance  specialists  responsible  for  identifying  complete  universes  of 
processes,  fully  documenting  process  proofings  and  reproofings  of  contractor 
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process  inputs,  and  reporting  progress  through  the  Quality  Assurance 
Management  Information  System.  The  system  of  accountability  needs  to  be 
reflected  in  quality  assurance  specialist  performance  standards. 

Identifying  Manufacturing  Processes.  Quality  assurance  specialists  at  the 
13  audit  sites  did  not  always  adequately  identify  all  of  the  manufacturing 
processes.  Quality  assurance  specialists  were  confused  and  inconsistent  when 
they  did  identify  manufacturing  processes.  Although  DCMC  instituted  IQUE  in 
1990,  quality  assurance  specialists  still  struggled  to  define  a  process  in  1994. 
The  DLA  "One  Book"  chapter  on  PROCAS  describes  processes  as  a  series  of 
tasks  leading  to  a  common  objective  and  satisfying  a  requirement,  such  as 
producing  an  estimate  or  making  an  item.  The  practices  used  for  identifying 
processes  at  all  13  audit  sites  did  not  include  a  documented  teamwork  approach 
involving  the  quality  assurance  specialist,  the  quality  assurance  specialist 
supervisor,  and  an  IQUE/PROCAS  coordinator.  The  following  examples  of 
process  identification  illustrate  the  confusion  that  exists  in  DCMC. 

o  The  uncertainty  of  what  constituted  a  manufacturing  process  at  the 
DPRO  McDonnell  Douglas  in  St.  Lnuis,  Missouri,  led  quality  assurance 
specialists  to  initially  identify  2,007  manufacturing  processes  applicable  to 
Defense  contracts.  In  1993  and  1994,  quality  assurance  specialists  continually 
reduced  the  number  of  manufacturing  processes  identified  as  they  modified  their 
interpretation  of  the  term  process  and  reduced  the  number  identified  to 
1,108  manufacturing  processes  as  of  June  1994. 

0  Quality  assurance  specialists  did  not  identify  any  manufacturing 
processes  at  Allied  Signal  in  Baltimore,  Maryland,  for  the  manufacture  of 
Patriot  missile  foses  and  at  UNISYS  in  Clearwater,  Florida,  for  the  manufacture 
of  memory  processors  and  other  computer  parts  for  use  on  the  Trident 
submarine.  TTierefore,  we  concluded  tlmt  IQUE  was  never  implemented  and 
quality  assurance  was  still  based  on  defect  identification  during  product 
inspections. 

o  Quality  assurance  specialists  identified  319  processes  for  the 
manufacture  of  underwater  cable  at  Simplex  in  Newington,  New  Hampshire. 
The  319  processes  were  actually  part  of  Simplex's  standard  practice  and 
inspection  manual  for  the  manufacture  of  the  underwater  cable.  The  quality 
assurance  specialists  had  not  identified  a  system  of  key  processes  to  address  as 
priority  actions. 

o  Quality  assurance  specialists  at  General  Electric  in  Lynn, 
Massachusetts,  initially  identified  about  5,000  processes  to  the  auditor  but  later 
during  the  audit  changed  their  minds  and  only  identified  41  processes  for  the 
F404  and  the  T700  engines.  The  41  processes  covered  broad  generic  areas  such 
as  welding.  The  uncertainty  at  the  General  Electric  plant  reflected  confusion  as 
to  what  constituted  a  process. 

o  At  Aerojet,  the  quality  assurance  specialists  provided  a  list  of 
232  processes  for  the  Delta  rocket  that  included  descriptions  of  manufacturing 
processes  described  simply  as  cleaning  (21  processes),  inspection 
(21  processes),  deburring  (12  processes),  and  X-ray  (8  processes).  The  quality 
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assurance  specialists  provided  no  other  supporting  descriptions  for  those 
62  processes.  The  unsupported  list  of  232  processes  demonstrated  an 
incomplete  implementation  of  IQUE  for  the  Delta  rocket. 

Appropriate  Level  of  Proofing  Processes.  Quality  assurance  specialists  at 
one  audit  site  did  not  demonstrate  that  they  understood  the  manufacturing 
operations  they  were  assigned  and  did  not  identify  the  appropriate  level  to  proof 
processes  to  ensure  that  DoD  can  rely  on  a  manufacturing  process.  At  DPRO 
Raytheon,  two  quality  assurance  specialists  assigned  to  the  missile  transmitter  of 
the  Patriot  missile  program  identified  nine  overall  manufacturing  processes. 
The  quality  assurance  specialists  maintained  they  had  proofed  the  overall 
processes  for  the  subsystem  assembly  of  the  missile  transmitter,  documenting 
the  proofing  with  a  flowchart  and  a  one-page  review  sheet  of  the  process  inputs. 
However,  the  quality  assurance  specialists  could  not  explain  the  transmitter 
subsystem  assembly  process  for  the  missile  transmitter.  The  quality  assurance 
specialists  requested  the  contractor  to  demonstrate  the  assembly  process.  While 
explaining  the  process,  the  contractor  identified  an  additional  20  subprocesses 
related  to  the  subsystem  assembly,  one  of  which  was  identified  as  a  critical 
process.  Because  they  were  not  fully  knowledgeable  about  the  manufacturing 
process,  the  quality  assurance  specialists  did  not  identify  any  of  the 
20  subprocesses  as  candidates  for  process  proofing  and  did  not  proof  the 
20  subprocesses. 

Documentation  For  Proofed  Manufacturing  Processes.  Documenting  an 
evaluation  of  a  manufacturing  process  is  basic  and  fundamental  to  process- 
oriented  quality  assurance.  Of  1,818  manufacturing  processes,  quality 
assurance  specialists  adequately  prepared  flowcharts  and  documented  proofed 
processes  for  only  99  processes  (5  percent).  The  remaining  1,777  (95  percent) 
manufacmring  processes  lacked  the  documentation  of  flowcharts  and 
descriptions  of  process  inputs  necessary  to  demonstrate  how  products  were 
made.  The  documentation,  if  any  at  all,  was  inadequate  and  could  rarely  be 
used  by  other  quality  assurance  specialists  to  determine  how  the  contractor 
process  was  supposed  to  function.  For  example,  a  quality  assurance  specialist  at 
Litton  Laser  in  Apopka,  Florida,  identified  and  reportedly  proofed 
8  manufacturing  processes  related  to  the  Low-Altimde  Navigation  Targeting 
Infra-Red  for  Night  or  LANTIRN  program.  A  checklist  of  process  inputs  and 
the  date  of  the  proofing  was  the  only  docmnentation  available.  We  reviewed 
the  proofing  documentation  with  the  quality  assurance  division  and  branch 
chiefs  and  they  determined  that  the  level  of  documentation  was  insufficient. 

At  2  of  the  13  audit  sites,  we  found  good  examples  of  professionally 
documented  processes  by  quality  assurance  specialists.  At  McDonnell  Douglas, 
the  documentation  for  support  processes  for  the  F/A-18,  the  AV-8B  Harrier  II, 
and  T-45  aircraft  were  professionally  prepared  and  documented.  In  the  second 
example,  quality  assurance  specialists  at  General  Electric  did  a  thorough  job  of 
documenting  a  few  manufacturing  processes  common  to  both  the  F404  and  the 
T700  engines.  Unfortunately,  the  overall  documentation  of  the  manufacturing 
processes  at  both  McDonnell  Douglas  and  General  Electric  was  incomplete. 
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Overall,  the  documentation  supporting  the  proofing  of  manufacturing  processes 
for  27  programs  was  inadequate  to  determine  what  actions  the  quality  assurance 
specialist  performed  to  ensure  or  prove  that  manufacturing  processes  functioned 
correctly.  The  quality  assurance  specialists  for  15  of  the  27  programs  did  not 
provide  any  documentation  to  demonstrate  when  and  how  the  manufacturing 
processes  were  proofed.  The  quality  assurance  specialists  for  the  remaining 
12  programs  could  only  provide  partial  documentation  for  the  proofing  of  all  of 
the  identified  manufacturing  processes.  Appendix  B  provides  a  summary  of  the 
manufacturing  process  identified  and  proofed  for  all  27  programs. 

Reproofing  Manufacturing  Processes.  Appendix  B  shows  that,  from 
1989  through  1992,  only  52  processes  were  proofed  for  the  27  programs.  Only 
5  of  the  52  processes  were  subsequently  reproofed,  even  though  each  contractor 
had  significant  changes  in  the  process  inputs.  We  concluded  that  reproofing  of 
processes  was  not  a  priority  for  quality  assurance  specialists,  even  though 
significant  changes  had  occurred  at  Defense  contractor  plants.  For  example,  at 
Aerojet,  the  entire  manufacturing  operation  for  the  Delta  rocket  was  moved  to  a 
single-cell  environment  during  1994.  The  move  resulted  in  new  equipment, 
new  operators,  and  a  new  environment;  however,  the  quality  assurance 
specialists  had  no  plans  to  reproof  the  processes. 

Reporting  Requirements.  The  DCMC  reporting  requirements  did  not  require 
quality  assurance  specialists  to  report  the  number  of  processes  and  critical 
processes  identified,  proofed,  and  reproofed  either  to  their  supervisors  of  to  the 
DCMDs.  DLA  set  up  the  Quality  Assurance  Management  Information  Systems 
data  base  to  monitor  the  workload  volume  of  field-level  activities.  Quality 
assurance  specialists  report  on  19  activity  indicators,  including  material  review 
board  decisions,  corrective  action  requests,  and  product  audits  through  the 
Quality  Assurance  Management  Information  Systems.  DCMC  did  not  include 
process  identification  and  proofing  as  a  reportable  activity  indicator.  DCMC 
needs  to  hold  quality  assurance  specialists  responsible  for  reporting  processes 
and  critical  processes  identified,  proofed,  and  reproofed  through  the  Quality 
Assurance  Management  Information  Systems. 

Performance  Standards.  DCMC  did  not  have  performance  criteria  to 
consistently  hold  quality  assurance  specialists  accountable  for  the  performance 
of  process  proofings.  Quality  assurance  specialists  position  descriptions  and 
performance  plans  were  not  consistent  throughout  DCMC  with  regard  to 
implementation  of  IQUE  and,  more  specifically,  the  proofing  of  contractor's 
processes. 

According  to  DLA  Regulation  1414.2,  "Civilian  Personnel  Position 
Classification  and  Appeals  Administration,"  March  9,  1992,  position 

descriptions  will  reflect  actual  work  performed  and  be  certified  for  accuracy  by 
supervisors.  However,  DLA  did  not  provide  the  necessary  standards  and 
guidelines  for  consistent  and  accurate  descriptions  of  the  work  that  was  needed 
to  effectively  implement  IQUE.  The  Office  of  Civilian  Personnel,  DLA,  was 
responsible  for  providing  standards  and  guidelines  through  agency  job 
guidelines  to  ensure  that  foe  major  duties  of  a  job  are  adequately  described  in 
local  positions. 
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An  agency  job  guideline  is  a  position  description  of  typical  work  performed  in  a 
given  occupation  at  a  given  grade  level  using  Office  of  Personnel  Management 
classification  criteria  applicable  to  that  agency  position.  DLA  did  not  update 
agency  job  guidelines,  including  responsibility  for  implementation  of  IQUE  or 
for  PROCAS.  DLA  eventually  chose  to  cancel  the  Agency  Job  Guidelines  for 
the  quality  assurance  specialist  1910  job  series  in  March  1993,  leaving  each 
DCMD  personnel  office  responsible  for  developing  individual  position 
descriptions  for  each  quality  assurance  specialist. 

The  position  descriptions  and  performance  plans  for  the  quality  assurance 
specialist  1910  job  series  from  each  of  the  DCMDs  contain  varying  descriptions 
of  IQUE  responsibilities  at  the  GS-9  and  GS-11  level.  Performance  standards  at 
the  DCMDs  ranged  from  a  general  standard  to  implement  IQUE  to  specific 
critical  job  element  standards  requiring  the  proofing  of  contractor  processes. 
GS-12  supervisory  standards  contained  only  general  wording  regarding  the 
implementation  of  IQUE  policies  arid  procedures.  Although  performance  plans 
generally  contain  wording  regarding  IQUE  implementation,  the  low  number  of 
processes  identified  and  identified  as  critical  and  the  lack  of  documentation 
supporting  processes  proofed  and  reproofed  is  firm  evidence  that  performance 
plans  are  not  designed  to  hold  quality  assurance  specialists  accountable  for 
accomplishing  fundamental  IQUE  tasks. 


Holding  Supervisors  Responsible 


DLAM  8200.5  provided  quality  assurance  specialists  wide  latitude  in  applying 
IQUE.  The  rationale  was  that  quality  assurance  specialist  supervisors  and 
higher  DCMC  management  should  not  stifle  initiative  through  direction  or 
control  of  quality  assurance  specialist  activities.  Supervisors  were  to  act  as 
leaders,  coaches,  and  mentors  for  the  quality  assurance  specialists.  We 
confirmed  that  quality  assurance  specialist  supervisors  generally  interpreted 
DLAM  8200.5  to  mean  that  supervisory  oversight  and  evaluation  of  quality 
assurance  specialist  implementation  of  IQUE  was  not  the  supervisor's  job. 

DCMC  did  not  hold  quality  assurance  specialist  supervisors  responsible  for 
reviewing  and  evaluating  the  identification  and  proofing  of  processes.  As  a 
result,  19  of  26  quality  assurance  specialist  supervisors  interviewed  did  not 
oversee,  evaluate,  monitor,  or  track  process  proofings  as  a  measure  of  the 
effectiveness  of  quality  assurance  specialist  implementation  of  IQUE.  None  of 
the  supervisors  interviewed  treated  process  proofing  as  a  priority  for  the 
implementation  of  IQUE  or  PROCAS.  DCMC  needs  to  hold  supervisors 
accountable  for  reviewing  and  evaluating  process-oriented  quality  assurance  and 
for  the  accuracy  of  the  quality  assurance  specialist-generated  information 
reported  through  the  Quality  Assurance  Management  Information  Systems. 
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Follow-up  on  IQUE  Reviews 


DCMC  Evaluation.  In  1992,  DCMC  authorized  a  number  of  staff  assistance 
visits  to  evaluate  the  progress  of  IQUE.  One  initiative,  "IQUE  Tomorrow, " 
was  chartered  to  obtain  feedback  from  quality  assurance  specialists  on  the 
implementation  of  IQUE.  Small  teams  of  2  or  3  DoD  and  DCMC  quality 
experts  visited  a  total  of  288  manufacturing  facilities,  including  DPROs  and 
quality  assurance  residencies  in  DCMAOs,  and  interviewed  more  than 
1,000  quality  assurance  specialists  from  February  3  through  July  24,  1992. 

The  "IQUE  Tomorrow"  visits  resulted  in  the  "Defense  Logistic  Agency  IQUE 
Tomorrow  Report  and  Action  Plan"  (the  IQUE  Tomorrow  Action  Plan), 
September  1992.  One  of  the  improvement  opportunities  identified  in  the  IQUE 
Tomorrow  Action  Plan  for  followup  was  fte  need  for  more  demonstrated 
techniques  of  process  proofings  and  data  collection  and  analysis.  We 
determined  that  very  little  progress  had  been  made  to  develop  more 
demonstrated  techniques  of  process  proofings  as  of  July  1,  1994,  for  the 
27  programs. 

In  addition,  subsequent  staff  assistance  visits  did  not  result  in  specific 
evaluations  and  recommendations  for  compulsory  action  to  complete  the  IQUE 
improvements  described  in  the  IQUE  Tomorrow  Action  Plan.  The  preparers  of 
the  IQUE  Tomorrow  Action  Plan  concluded  that  the  DCMDs  were  making 
good-faith  efforts  to  implement  IQUE  and  the  implementation  was  generally 
proceeding  at  a  satisfactory  rate  in  most  facilities.  Our  audit  did  not  confirm 
those  conclusions.  We  asked  to  review  the  workpapers  supporting  the  IQUE 
Tomorrow  Action  Plan,  and  we  were  told  that  DCMC  did  not  have  workpapers 
to  support  the  report.  DCMC  needs  to  document  the  measurement  of  its  efforts 
to  review  contractor  quality  control  processes. 

Validating  the  Progress  Made  to  Proof  Contractor  Quality  Control 
Processes.  The  success  of  PROCAS  on  quality  assurance  functions  is  based  on 
the  complete  proofing  of  contractors'  quality  control  processes.  Validating 
contractors'  quality  control  processes  would  allow  for  more  efficient  use  of 
quality  assurance  resources  by  identifying  and  targeting  problem  areas  and 
should  result  in  lower  inspection  requirements  and,  ultimately,  in  fewer  quality 
assurance  specialists  in  contractor  plants.  Reducing  the  number  of  quality 
assurance  specialists  at  Defense  contractor  plants  when  warranted  is  a  worthy 
goal.  Unfortunately,  DCMC  does  not  have  an  objective  measurement  to 
determine  the  extent  to  which  contractors'  processes  are  valid. 

Inspector  General,  DoD,  Report  No.  94-079  recommended  that  DLA  revise  the 
"Action  Plan  for  Continuously  Improving  the  (Quality  of  Spare  and  Repair 
Parts,"  May  1990,  by  adding  an  initiative  to  measure  the  extent  to  which  each 
DoD  contractor's  quality  control  processes  are  reviewed  and  validated.  DLA 
nonconcurred  with  the  recommendation,  stating  that  PROCAS  was  replacing 
IQUE  to  facilitate  communications  among  contract  administration  participants. 
Our  audit  shows  that  PROCAS  alone  will  have  little  effect  on  how  well 
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individual  quality  assurance  specialists  perform  their  jobs.  DCMC  needs  to 
determine  the  extent  of  actual  quality  assurance  specialist  reviews  and  validation 
of  contractor  quality  control  processes. 


Realistic  Risk  Assessments  Through  Performance 
Measurement 


Internal  Management  Control  Program.  From  1989  to  1992,  DLA  reported 
nonconforming  material  in  the  DoD  Supply  System  as  a  material  weakness  as 
part  of  its  annual  statement  of  assurance  in  accordance  with  DoD  Directive 
5010.38.  One  of  the  corrective  actions  was  to  implement  IQUE  to  include 
prime  contractor  control  of  subcontractor  material.  In  1993,  DLA  reported  that 
the  material  weakness  was  corrected.  However,  IQUE  was  never  adequately 
implemented. 

The  1993  overall  quality  assurance  risk  assessment  prepared  by  the 
three  DCMDs  concluded  that  the  risk  of  receiving  poor  quality  products  through 
IQUE  was  low.  The  basis  for  that  conclusion  was  the  perceived  success  of 
IQUE  in  the  DCMAOs  and  DPROs.  However,  DLA  did  not  have  sufficient 
basis  for  concluding  that  the  IQUE  program  was  a  success  because  IQUE  was 
not  evaluated  or  measured  in  terms  of  process  proofings  or  other  process- 
oriented  quality  assurance  indicators  during  1993  or  in  prior  years. 

In  our  opinion,  insufficient  basis  exists  for  assigning  a  low  risk  to  the  quality 
function  when  DCMC  has  not  made  an  adequate  assessment  of  such  risks  based 
on  thorough  IQUE  evaluations  of  contractor  processes.  In  addition,  DCMC 
needs  to  evaluate  process-oriented  quality  assurance  in  accordance  with  criteria 
that  measure  the  execution  of  IQUE  and  PROCAS  in  terms  of  the  extent  that 
each  DoD  supplier's  quality-related  processes  are  reviewed  and  validated. 

Performance  Measurement.  Public  Law  103-62,  "Government  Performance 
and  Results  Act  of  1993,"  August  3,  1993  (the  Act),  was  enacted  to  improve  the 
efficiency  and  effectiveness  of  Federal  programs  by  requiring  systems  that  set 
performance  goals  and  measure  results  in  quantifiable  forms.  The  Act 
emphasizes  the  accountability  of  Federal  agencies  for  systematically  achieving 
program  results.  DoD  specifically  identified  DLA  to  lead  a  pilot  project  for 
compliance  with  the  Act's  performance  measurement  reporting  requirements  in 
FYs  1994,  1995,  and  1996.  As  part  of  the  annual  reporting,  DLA  was 
required  to  present  specific  measures  and  targets  as  outcome  indicators  for  each 
program  activity.  The  indicators  were  intended  to  permit  a  future  assessment  of 
program  results  against  the  intended  purpose  of  each  program. 

The  DLA  FY  1995  Performance  Plan  indicator  for  quality  assurance  was 
"product  audits  that  result  in  requests  for  corrective  action. "  We  believe  that 
DCMC  chose  this  indicator  for  the  DLA  Performance  Plan  because  data  on 
product  audits  were  available  as  reportable  entities  in  the  existing  Quality 
Assurance  Management  Information  System.  Unfortunately,  poduct  audits  do 
not  establish  a  meaningful  performance  measure  or  establish  a  target  for 
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successful  implementation  of  the  PROCAS  initiative  as  described  in 
DLA  directives.  More  meaningful  performance  measures  would  address  the 
number  of  critical  processes  identified,  realistic  goals  for  completion  of 
proofing  processes,  and  the  number  of  process  proofings  performed.  DCMC 
does  not  have  goals  for  documenting  process-oriented  quality  assurance  through 
process  proofings  or  for  measuring  the  effort  of  quality  assurance  personnel 
responsible  for  performing  quality  assurance.  DCMC  does  not  have  a  system  of 
accountability  for  achieving  effective  process-oriented  quality  assurance. 

Employing  Process-Oriented  Quality  Assurance.  Quality  assurance 
specialists  did  not  effectively  employ  process-oriented  quality  assurance  to 
establish  a  basis  for  substituting  process  proofing  and  product  audits  for 
mandatory  inspections.  Mandatory  inspections  are  detailed  inspections  of 
product  conformance  to  product  specifications.  These  inspections  are  laborious 
and  time  consuming  and  are  intended  to  identify  any  possible  latent  defects. 
Buying  commands  or  program  offices  often  impose  mandatory  inspections  on 
contract  quality  assurance  for  mission  critical  items  and  for  munitions.  Buying 
commands  are  frequently  unwilling  to  use  IQUE  process-oriented  quality 
assurance,  even  for  items  that  never  or  rarely  experience  manufacturing  failures. 
Unfortunately,  quality  assurance  specialists  at  some  locations  have  not 
implemented  IQUE,  thus,  the  buying  commands  could  not  see  the  advantages  of 
using  a  combination  of  process  proofing  and  product  audits  instead  of 
mandatory  inspections.  The  following  are  examples  for  which  quality  assurance 
specialists  at  DCMC  quality  assurance  residencies  did  not  implement  IQUE 
because  all  of  the  quality  assurance  work  involved  mandatory  inspections. 

o  UNISYS  in  Clearwater,  Florida,  manufactures  computer  parts  for  the 
Trident  missile.  The  buying  command  imposed  100-percent  mandatory 
inspection  on  every  part  manufactured  by  UNISYS.  The  quality  assurance 
specialist  assigned  to  the  Trident  program  requested  relief  from  the  buying 
command  in  providing  100-percent  mandatory  inspection  and  instead  suggested 
labor-saving  IQUE  quality  assurance  coverage  of  Trident  missile  parts.  The 
buying  command  rejected  the  request.  We  examined  the  implementation  of 
IQUE  at  UNISYS  and  determined  that  the  quality  assurance  specialist  had  not 
proofed  the  manufacturing  processes  and  implemented  IQUE.  DCMC  quality 
assurance  personnel  needed  to  proof  the  UNISYS  manufacturing  processes  for 
the  Trident  before  asking  the  Navy  to  rely  on  the  contractor's  quality  system  to 
provide  conforming  products. 

o  A  similar  situation  existed  at  Honeywell  in  Clearwater,  Florida.  The 
Navy  requested  100-percent  mandatory  inspection  on  parts  manufactured  for 
MK-6  guidance  system  used  in  the  Trident  missile.  From  September  1991 
through  April  1994,  the  quality  assurance  specialist  performed  358  mandatory 
inspections  on  parts  manufactured  for  the  MK-6  program.  Of  the 
358  inspections,  357  (99.8  percent)  passed  inspection  with  no  defects 
discovered.  The  quality  assurance  specialist  stated  that  he  was  not  proofing 
processes  for  the  MK-6  program  because  he  was  too  busy  performing 
100-percent  mandatory  inspections  as  the  Navy  requested.  If  the  DCMAO 
Clearwater  had  devoted  resources  to  proof  processes  for  the  MK-6  program,  the 
quality  assurance  specialist  would  have  established  a  basis  for  requesting  that  the 
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Navy  reconsider  its  request  for  100-percent  mandatory  inspections  and  for 
asking  the  Navy  to  accept  quality  assurance  based  on  process  proofing  and 
product  audits. 


Conclusion 


DCMC  has  not  managed  the  quality  assurance  workforce  effectively  to 
implement  the  primary  quality  assurance  program,  IQUE.  Quality  assurance 
specialists  have  not  adequately  documented  their  work  to  provide  a  record  of 
contractors'  ability  to  manufacture  products  through  reliable  processes.  DCMC 
is  facing  a  changing  environment  without  the  b^enefit  of  a  well-documented 
record  of  the  reliability  of  Defense  suppliers  and  the  degree  of  risk  associated 
with  each  supplier.  DCMC  needs  to  institute  an  effective  quality  assurance 
program  that  includes  effective  performance  measurement  and  accountability  for 
effective  performance. 


Management  Comments  on  the  Finding  and  Audit  Response 


DLA  Comments.  DLA  nonconcurred  with  Finding  A,  stating  that  the 
Inspector  General,  DoD,  did  not  provide  sufficient  evidence  for  the  finding  that 
DCMC  did  not  effectively  manage  quality  assurance  specialists,  and  that  the 
IQUE  and  PROCAS  methodologies  were  not  effectively  managed  at  the  13  audit 
sites.  DLA  disagreed  with  the  Inspector  General,  DoD,  position  relating  to  the 
level  of  documentation  that  should  be  maintained.  In  addition,  the  Inspector 
General,  DoD,  offered  no  evidence  concerning  defective  product  accepted  at  the 
13  audit  sites  to  support  statements  that  DCMC  quality  assurance  is  inadequate. 
DLA  also  stated  that  the  improvement  scenario  offered  in  the  report 
recommendations  were  ill  founded,  because  of  the  "critical  process"  concept 
applied  by  the  audit  teams  and  the  auditors'  misinterpretation  of  what 
constituted  sufficient  and  proper  process  proofing  documentation. 

Audit  Response.  The  DLA  refusal  to  recognize  the  material  internal  control 
problems  that  exist  in  the  system  for  product  quality  assvuance  is  the  basis  for 
the  ineffective  management  of  quality  assurance  specialists.  The  problem  of 
nonconforming  products  accepted  into  the  Defense  supply  system  is  well 
documented  in  prior  audit  reports.  An  example  of  the  continuing  serious 
problem  with  nonconforming  products  was  demonstrated  in  the  February  1995 
debarments  of  two  Defense  contractors  that  supplied  defective  aerospace, 
automotive  and  missile  components,  and  spare  parts.  The  defective  products 
entering  the  Defense  supply  system  adversely  effects  the  utility  of  military  end 
items,  impacts  readiness,  and  continues  to  demonstrate  the  ineffectiveness  of 
DLA  quality  assurance  efforts. 

We  agree  with  the  DCMC  concept  to  emphasize  critical  processes  over 
noncritical  processes.  We  do  not  agree  that  using  the  critical  process  concept 
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makes  our  report  recommendations  ill  founded.  The  criteria  used  in  the  audit 
for  adequate  documentation  of  process  proofing  was  less  stringent  than  the  level 
of  documentation  that  DCMC  included  in  the  "One  Book"  after  we  completed 
our  audit.  If  the  "One  Book"  criteria  had  been  available  during  the  audit,  the 
number  of  process  proofings  in  Appendix  B  that  were  adequately  documented 
would  decline  from  99  to  0.  We  stopped  the  audit  after  13  sites,  because  none 
of  the  13  sites  were  adequately  documenting  the  IQUE  process  proofing 
concept.  During  the  audit,  we  had  previously  identified  at  least  10  other  sites  in 
the  Western  District  that  likely  would  have  reflected  a  comparable  lack  of 
documentation.  However,  we  had  sufficient  evidence  to  demonstrate  that 
quality  assurance  specialists  were  not  effectively  managed  after  13  sites,  so  we 
concluded  the  audit. 


Recommendations,  Management  Comments,  and 
Audit  Response 


Revised  Recommendations.  Based  on  management  comments,  we  revised 
Recommendations  2.,  3.,  4.,  and  5.  to  connect  the  recommendations  to  Public 
Law  103-62,  "Government  Performance  and  Results  Act  of  1993,"  and  to 
afford  DLA  the  opportunity  to  describe  how  it  plans  to  comply  with  the  public 
law  by  1997  as  it  applies  with  our  recommendations. 

We  recommend  that  the  Director,  Defense  Logistics  Agency: 

1.  Define  in  Defense  Logistics  Agency  Manual  8000.5,  "One  Book,"  (draft) 
October  1994,  the  term  "critical  manufacturing  process"  so  that  the 
meaning  and  intent  of  the  term  is  clear. 

DLA  Comments.  DLA  concurred  and  stated  that  the  following  definition 
would  be  included  in  the  "One  Book."  A  process  is  critical  if  it  significantly 
affects  the  cost,  schedule,  or  technical  performance  of  the  product. 

2.  Establish  performance  indicators  to  measure  the  performance  of  quality 
assurance  specialists  for: 

a.  Identifying  a  complete  universe  of  processes  and  subprocesses  to 
a  level  of  detail  that  satisfactorily  describes  the  contractor's  sequence  of 
processes  for  producing  conforming  products  in  accordance  with  each 
contract  assigned. 

b.  Identifying  critical  processes  and  documenting  why  the  process 
is  critical. 


c.  Fully  documenting  process  proofings  of  contractor  process  inputs 
including  equipment,  materials,  people,  methods,  and  environmental 
controls,  and  either  flowcharting  extensive  or  complex  processes  or,  for 
simple  processes,  listing  the  sequence  of  applicable  operations. 
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d.  Fully  documenting  process  reproofings  and  the  changes  in 
contractor  processes  every  2  years.  If  tiie  contractor  processes  do  not 
change,  document  the  fact  that  no  changes  had  occurred. 

e.  Reporting  through  the  Quality  Assurance  Management 
Information  System  processes  and  critical  processes  identified,  proofed, 
and  reproofed. 

DLA  Comments.  DLA  nonconcurred  with  the  draft  report  recommendations 
regarding  the  need  to  identify  the  universe  of  processes  or  to  require  a  proofing 
of  processes  every  2  years.  However,  DCMC  recognized  the  ne^  for  adequate 
documentation  of  process  proofing  and  the  need  to  reproof  processes. 
Furthermore,  DCMC  described  its  flexible  system  of  management  that  allows 
for  effectively  responding  to  significant  events  and  that  provides  the  DCMAOs 
the  flexibility  to  structure  effective  programs.  DLA  further  stated  that  the  "One 
Book"  requires  DCMC  PROCAS  teams  to  document  process  proofing, 
including  the  techniques  used,  the  process  inputs  and  outputs  proofed,  where  the 
proofing  was  performed,  who  performed  the  proofing,  proofing  results,  and 
actions  taken  based  on  proofing  results. 

Audit  Response.  After  4  years  of  IQUE,  DCMC  quality  assurance  specialists 
could  not  demonstrate  that  quality  controls  were  in  place  because  they  had  not 
adequately  documented  process  proofings.  Understanding  the  universes  of 
processes  will  enable  quality  assurance  specialists  to  identify  the  critical 
processes  that  require  immediate  attention.  The  "One  Book"  requirements  for 
documentation  that  were  instituted  after  we  conducted  our  audit,  more  than 
satisfy  the  professional  documentation  standards  that  we  had  in  mind.  If  we  had 
applied  those  standards  to  the  1,818  processes  we  reviewed  for  the  27  programs 
identified  in  Appendix  B,  the  number  of  processes  proofed  would  decline  from 
99  to  0.  DCMC  needs  an  effective  system  for  executing  the  innovative  quality 
assurance  programs  that  it  has  developed.  We  have  revised  the 
recommendations,  accordingly.  Therefore,  we  request  that  DLA  reconsider  its 
position  and  provide  additional  comments  to  die  ^al  report  describing  how  it 
intends  to  measure  the  performance  of  quality  assurance  specialists  for  the 
documentation  of  proofing  critical  processes. 

3.  Establish  perfonnance  indicators  to  measure  the  performance  of  the 
supervisors  of  quality  assurance  specialists  responsible  for  reviewing  and 
evaluating: 

a.  The  adequacy  of  the  universes  of  processes  identified. 

b.  The  adequacy  of  documentation  supporting  processes  proofed 
and  reproofed. 

c.  The  accuracy  of  the  information  reported  by  quality  assurance 
specialists  through  the  Quality  Assurance  Management  Information 
System. 
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DLA  Conunents.  DLA  nonconcurred  with  the  draft  recommendations  for 
establishing  a  system  of  accountability  to  make  supervisors  responsible  for 
reviewing  and  evaluating  the  work  of  quality  assurance  specialists.  DLA  stated 
that  it  did  not  believe  identifying  the  universe  of  processes  was  always  n^essary 
and  DCMC  supervisors  were  ^eady  responsible  for  guiding,  supporting,  and 
managing  the  work  of  their  employees. 

Audit  Response.  DCMC  needs  an  effective  system  for  executing  the 
innovative  quality  assurance  programs  that  it  has  developed.  As  stated  in  the 
finding,  19  of  26  supervisors  did  not  oversee,  evaluate,  and  review  Ae 
adequacy  of  process  proofings.  The  lack  of  supervisory  review  and  evaluation 
is  a  primary  cause  for  the  lack  of  effective  management.  We  request  that  DLA 
reconsider  its  position  and  provide  additional  comments  describing  how  it 
intends  to  measure  the  performance  of  its  quality  assurance  speci^st 
supervisors  for  the  review  and  evaluation  of  the  documentation  of  critical 
process  proofings. 

4.  Establish  performance  indicators  to  measure  the  extent  to  which  each 
DoD  contractor's  quality  control  processes  are  reviewed  and  validated, 
consistent  with  Recommendation  B.S.h.  in  Inspector  General,  DoD,  Report 
No.  94-079,  "DoD  Components  Implementing  Action  Flans  for  Improving 
the  Quality  of  Spare  Parts,"  April  12, 1994. 

DLA  Comments.  DLA  nonconcurred  with  revising  the  DLA  Action  Plan  for 
Continuously  Improving  the  Quality  of  Spare  Parts,  stating  that  it  did  not 
believe  tiiat  merely  counting  tasks  was  a  good  measure  of  performance.  DLA 
did  not  believe  it  was  always  necessary  to  count  the  processes  in  the  universe. 
Therefore,  it  was  inappropriate  to  use  that  universe  as  a  baseline  against  which 
process  review  and  validation  efforts  are  measured.  Furthermore,  the  DLA 
evolving  assessment  methodology  focuses  on  local  responsibility  and 
accountability,  using  the  process  outlined  in  the  "One  Book."  The  process 
evaluates  the  effectiveness  of  tiie  local  approach,  deployment,  and  results,  rather 
than  just  compliance  with  mandated  documentation  requirements. 

Audit  Response.  This  recommendation  was  previously  made  in  Inspector 
General,  DoD,  Report  No.  94-079.  DLA  nonconcurred  with  the 
recommendation  in  that  report,  stating  that  PROCAS  would  remedy  the  problem 
by  facilitating  the  communications  between  contract  administration  participants. 
We  accepted  the  DLA  remedy  at  that  time  because  we  were  reviewing 
implementation  of  PR(X!AS  in  this  audit.  DCMC  needs  an  effective  system  for 
measuring  execution  of  the  innovative  quality  assurance  programs  that  it  has 
developed.  We  have  revised  the  recommendation  and  request  that  DLA 
describe  how  it  plans  to  measure  the  performance  of  its  quality  assurance 
specialists  for  reviewing  and  validating  contractor  quality  control  processes. 
We  request  that  DLA  reconsider  its  position  and  provide  additional  comments  to 
the  final  report. 


22 


Finding  A.  Managing  the  Quality  Assurance  Workforce 


5.  Establish  in  the  Internal  Management  Control  Program: 

a.  Requirements  to  establish  management  controls  to  minimize  the 
risks  associated  with  process  oriented  quality  assurance. 

b.  Quality  assurance  as  a  high  risk  area  until  quality  assurance 
implementation  is  documented  according  to  the  criteria  established  in  the 
Defense  Logistics  Agency  Manual  8000.5  "One  Book." 

DLA  Comments.  DLA  nonconcurred  with  the  draft  recommendation  to 
establish  a  requirement  in  the  Internal  Management  Control  Program  to  evaluate 
process  oriented  quality  assurance  in  accordance  with  criteria  that  measures  the 
extent  that  each  DoD  suppliers  quality  controls  are  reviewed.  DLA  stated  that 
merely  counting  the  tasla  was  not  a  good  measure  of  performance.  DLA  did 
not  believe  it  was  always  necessary  to  count  the  processes  in  the  universe. 
Therefore,  it  was  inappropriate  to  use  that  universe  as  a  baseline  against  which 
process  review  and  validation  efforts  are  measured.  Further,  the  DLA  evolvmg 
assessment  methodology  focuses  on  local  responsibility  and  accountability, 
using  the  process  outlined  in  the  "One  Book."  The  process  evaluates  the 
effectiveness  of  (he  local  approach,  deployment,  and  results,  rather  than  just 
compliance  with  mandated  documentation  requirements. 

Audit  Response.  The  recommendations  in  the  report  were  designed  to  obtain 
specific  measurable  action  from  the  quality  assurance  workforce  and  the  DCMC 
managers.  DLA  apparenfly  does  not  desire  to  measure  how  the  workforce 
implements  quality  assurance  programs.  We  believe  that  DLA  needs  to 
incorporate  the  review  and  measurement  of  quality  assurance  as  part  of  the 
management  control  program  and  assess  the  risks  related  to  changing  the  quality 
assurance  program.  DLA  has  delegated  assessments  of  the  quality  assurance 
program  to  local  levels.  This  finding  shows  that  the  local  levels  did  not 
implement  the  DLA  quality  assurance  program.  DLA  does  not  regard  not 
implementing  the  quality  assurance  program  at  the  local  DCMC  levels  as  a 
problem.  We  have  revised  the  recommendation.  We  request  that  DLA 
reconsider  its  position  and  provide  additional  comments  to  the  final  report. 

Management  Comments  Required.  See  Table  2.  at  the  end  of  Finding  B.  for 
a  summary  of  management  comments  required. 
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DCMC  did  not  know  how  many  quality  assurance  specialists  were 
needed  to  perform  the  quality  assurance  mission  because  DCMC  does 
not  have  an  effective  method  for  determining  quality  assurance  staffing 
levels.  Specifically,  DCMC  does  not  analyze  the  actual  work  required 
to  perform  process-oriented  quality  assurance  required  for  IQUE  and 
PROCAS.  In  addition,  DCMC  routinely  adjusts  the  previous  years' 
staffing  levels  downward  relying  primarily  on  normal  attrition  to  fit 
reduced  funding.  As  a  result,  DCMC  did  not  have  an  adequate  basis  for 
justifying  aimual  funding  requests  for  quality  assurance  staffing  required 
to  perform  the  quality  assurance  mission  and  for  determining  the  actual 
imbalances  in  quality  assurance  staffing  levels. 


Quality  Assurance  Staffing 


Since  1991,  DCMC  has  streamlined  the  DCMDs  and,  since  October  1991,  the 
number  of  DCMC  quality  assurance  specialists  decreased  from  7,125  to 
5,567  (20  percent)  as  of  October  1994.  During  1994,  DCMC  reduced  the 
number  of  DCMDs  from  five  to  three.  Also,  during  1994,  DCMC  reduced  the 
number  of  quality  assurance  specialists  from  6,183  to  5,567.  As  of 
October  1994,  DCMC  employed  5,  567  quality  assurance  specialists,  at  a  cost 
of  $295.2  million  as  shown  in  the  following  table. 


Table  1.  DCMD  Quality  Assurance  Specialist  Staffing  and  Labor  Costs 


(as  of  October  1994) 

DCMD 

Number  of 

Oualitv  Assurance  Specialists 

Labor  Costs 

Northeast 

2,304 

$127,158,860 

West 

2,082 

103,662,788 

South 

1,181 

64.445.882 

Total 

5.567 

$295,267,530 

The  number  of  quality  assurance  specialists  is  expected  to  decrease  by  about 
4  percent  in  1995  (5,344  quality  assurance  specialists)  and  1996  (5,131  quality 
assurance  specialists). 

DCMC  used  historical  costs  related  to  two  elements,  quality  assurance  and 
mandatory  inspections,  as  reported  through  the  DLA  Unit  Cost  System  to 
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determine  the  staffing  levels  for  quality  assurance.  The  cost  of  quality 
assurance  primarily  represents  the  cost  of  the  quality  assurance  specialists'  time 
spent  on  all  quality  assurance  activities  other  tihan  mandatory  inspections.  The 
cost  of  mandatory  inspections  primarily  represents  the  cost  of  the  time  spent  on 
the  inspections  of  critical  items  and  other  items  called  for  in  quality  assurance 
letters  of  instruction  from  contracting  officers  and  from  program  managers.  A 
description  of  the  DLA  Unit  Cost  System  is  in  Appendix  E. 


Analyzing  and  Documenting  Quality  Assurance 
Workload  Requirements 


DCMC  does  not  know  whether  they  have  too  few  or  too  many  quality  assurance 
specialists  to  accomplish  the  quality  assurance  mission.  DCMC  did  not  analyze 
the  quality  assurance  workload  to  determine  what  part  of  the  quality  assurance 
mission  was  not  effectively  accomplished. 

DCMC  quality  assurance  personnel  did  not  document  workload  for 
accomplishing  die  basic  quality  assurance  tasks  for  any  of  the  104  contracts  we 
reviewed.  The  contracts,  valued  at  $10.8  billion,  involved  27  weapon  system 
and  support  programs  and  were  managed  by  65  quality  assurance  specialists 
assigned  to  6DCMAOs  and  3  DPROs.  A  detailed  description  is  in 
Appendix  C. 

Documenting  quality  assurance  workload  requirements  should  begin  with  an 
analysis  of  the  quality  assurance  tasks  required  for  each  contract.  Quality 
assurance  specialists  reviewed  their  assigned  contracts  to  identify  the  military 
specification  requirements  listed  in  the  contracts.  However,  quality  assurance 
specialists  did  not  analyze  the  actual  quality  assurance  work  needed  to  provide 
reasonable  assurance  that  only  conforming  products  were  accepted. 

DCMC  lacked  quality  assurance  workload  planning  at  successive  levels  of 
administration:  from  the  DCMDs  to  the  DCMAOs,  to  the  quality  assurance 
residencies  at  contractor  facilities,  to  the  supervisors  of  quality  assurance 
specialists  for  the  27  programs.  The  Government  quality  assurance  personnel 
did  not  adequately  review  the  contracts  to  determine  the  work  required  to  proof 
and  revalidate  contractor  processes,  to  perform  mandatory  inspections,  to 
perform  quality  audits,  to  investigate  Product  Quality  Deficiency  Reports,  to 
review  minor  nonconforming  material  presented  by  contractors  for  acceptance, 
and  to  perform  the  delegated  quality  assurance  related  to  subcontracts. 

Planning  for  Process-Oriented  Quality  Assurance.  The  quality  assurance 
supervisors,  representatives,  and  specialists  responsible  for  all  104  contracts 
lacked  documentation  to  establish  a  basis  for  determining  the  number  of  quality 
assurance  specialists  needed  to  identify,  proof,  and  revalidate  applicable 
contractor  processes.  The  primary  workload  tasks  of  a  quality  assurance 
specialist  are  the  examination  of  processes  as  part  of  IQUE  and  PROCAS. 
DLA  Manual  8200.5  provided  guidance  for  quality  assurance  specialists  to 
"proof"  contractor  manufacturing  processes  through  stages  to  assess  how 
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effectively  and  accurately  the  contractor  incorporates  personnel,  materials, 
methods,  equipment,  and  environment  to  satisfy  contract  requirements  and 
produce  conforming  products.  However,  as  stated  in  Finding  A,  quality 
assurance  specialists  did  not  identify  contractor  processes  and  did  not  adequately 
docmnent  Ae  proofing  of  contractor  processes.  Quality  assurance  specialists 
frequently  performed  product  audits  (examinations  of  process  outcomes)  after 
the  contractor  tested  products  to  gain  assurance  of  consistent  performance. 
Quality  assurance  specialists  did  not  document  the  estimated  number  of  product 
audits  that  were  needed  for  current  contracts. 

DCMC  should  require  quality  assurance  specialists  to  document  estimated 
quality  assurance  workload  on  the  Quality  Assurance  Management  Information 
Systems,  and  supervisors  should  review  and  certify  the  quality  assurance 
specialist  workload  estimates  reported  on  the  Quality  Assurance  Management 
Information  Systems. 

Planning  for  Mandatory  Inspections.  The  cost  of  mandatory  inspections  are 
reported  through  the  DLA  Unit  Cost  System,  but  the  DCMAOs  and  the  DPROs 
do  not  evaluate  the  projected  mandatory  inspection  workload  for  staffing 
planning  purposes.  The  DLA  Unit  Cost  System  is  described  in  Appendix  E. 

Of  the  104  contracts,  37  contracts,  valued  at  $9  billion,  included  requirements 
for  mandatory  inspections.  (Quality  assurance  persoimel  did  not  document  the 
basis  for  determining  the  quality  assurance  resources  needed  to  identify,  plan 
and  schedule,  and  perform  mandatory  inspections. 

Quality  assurance  specialists  should  have  an  active,  documented  workload  plan 
Aat  lists  IQUE  projects  that  the  quality  assurance  specialists  are  continually 
working.  However,  quality  assurance  specialists  at  6  of  13  audit  sites  told  us 
that  they  often  waited  for  contractors  to  notify  them  of  mandatory  inspections. 
None  of  the  quality  assurance  specialists  interviewed  were  attempting  to  identify 
processes,  to  proof  processes,  or  to  perform  product  audits  while  they  waited 
for  the  contractor  to  notify  them  Aat  they  were  needed  for  a  mandatory 
inspection. 

Planning  for  Other  Quality  Assurance  Work.  In  addition  to  process-oripted 
quality  assurance  tasks,  quality  assurance  specialists  are  often  assigned 
additional  work  such  as  conducting  material  reviews  and  investigating  Product 
Quality  Deficiency  Reports.  Depending  on  the  contractor  quality  controls,  a 
quality  assurance  specialist  may  have  a  significant  amount  of  work  assigned  in 
material  reviews  and  investigations.  Planning  for  other  work  such  as  material 
reviews  and  investigations  of  Product  Quality  Deficiency  Reports  needs  to  be 
documented  based  on  historical  experience. 

Planning  for  Subcontractor  Quality  Assurance.  Quality  assurance  specialists 
at  four  quality  assurance  residencies,  one  each  at  DCMAOs  Clearwater  and 
Orlando  and  two  at  DCMAO  Baltimore,  were  not  given  information  on 
subcontract  awards  in  a  timely  manner.  If  DCMD  quality  assurance  specialist 
supervisors  do  not  know  about  pending  work,  then  they  will  not  be  able  to  plan 
for  appropriate  numbers  of  quality  assurance  specialists.  Qualify  assurance 
specialists  were  not  always  aware  of  the  existence  of  subcontracts  until 
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subcontractor  persoimel  notified  them  that  the  subcontractor  was  read;^  for  the 
quality  assurance  specialist  to  perform  a  mandatory  Government  inspection  or  to 
acfcept  a  product.  Quality  assurance  specialists  were  unaware  of  the 
subcontractor  quality  assurance  work  because  the  assignment  of  quality 
assurance  work  from  the  contract  administration  office  supporting  the  prime 
contractor  to  the  contract  administration  office  supporting  the  subcontractor  does 
not  always  work  efficiently. 


Reduced  Funding  Levels  and  Attrition 


DCMC  decreased  the  overall  funding  for  every  DCMD  during  1992  and  1993; 
therefore,  personnel  reductions  were  necessary.  The  reductions  in  quality 
assurance  specialists  were  obtained  from  the  acmal  attrition  that  occurred  within 
the  DCMDs.  The  number  of  quality  assurance  specialists  needed  to  complete 
process  proofings  and  product  audits,  the  principal  tasks  associated  with 
process-oriented  quality  assurance,  was  not  a  primary  consideration  in  the 
funding  decisions.  DCMC  should  use  documented  workload  estimates  of 
process-oriented  quality  assurance  tasks  as  the  basis  for  budget  requests. 
DCMC  provided  DCMD  commanders  the  authority  for  determining  staffing  at 
DPROs  and  DCMAOs.  The  DCMD  commanders  used  past  staffing  levels  as 
their  guide  and  made  downward  adjustments  of  quality  assurance  staffing  to 
align  with  the  reduced  funding  that  was  provided  to  the  DCMDs  from  DCMC. 

The  following  are  examples  of  staffing  through  reduced  funding. 

Aerojet.  In  May  1992,  DCMD  West  performed  a  resource  review  at 
Aerojet.  The  review  team  recommended  that  staffing  remain  at  the  13  quality 
assurance  specialist  positions  on  board  through  the  third  and  fourth  quarters  of 
FY  1992.  The  review  team  recommendation  was  based  on  the  need  to  refocus 
quality  assurance  work  and  implement  IQUE.  For  FY  1993,  the  resource 
review  team  recommended  reducing  the  13  quality  assurance  specialist  positions 
to  9  to  reflect  the  anticipated  progress  in  implementing  IQUE.  After 
April  1993,  the  staff  was  reduced  to  9;  however,  IQUE  was  not  implemented. 
Of  722  contractor  processes  identified,  the  13  quality  assurance  specialists  only 
proofed  21  processes  by  April  1994.  The  quality  assurance  specialists  still  had 
701  processes  to  proof.  In  addition,  the  Aerojet  residency  chief  assigned  two  of 
the  remaining  nine  quality  assurance  specialists  to  work  on  other  contract 
administration  duties  instead  of  quality  assurance,  leaving  only  seven  quality 
assurance  specialists  to  perform  quality  assurance  work  at  the  Aerojet  residency. 
The  Aerojet  residency  chief  did  not  analyze  the  work  required,  and  he  reported 
to  DCMD  West  that  he  did  not  know  what  his  staffing  should  be. 

McDonnell  Douglas.  DCMC  demonstrated  a  practice  of  staffing 
according  to  attrition  at  the  DPRO  McDonnell  Douglas.  McDonnell  Douglas 
consolidated  manufacturing  operations  and  transferred  workload  into  the 
McDonnell  Douglas,  St.  Louis,  Missouri,  facility.  The  transfer  resulted  in 
additional  quality  assurance  work  for  the  DPRO  McDonnell  Douglas. 
However,  the  additional  work  did  not  result  in  the  stabilization  of  quality 
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assurance  staffing  for  the  DPRO  McDonnell  Douglas.  In  1992,  the  DPRO 
employed  87  quality  assurance  specialists;  in  1993,  the  number  of  quality 
assurance  specialists  declined  to  79,  and  then  to  71  in  1994.  DCMC  did  not 
allow  DPRO  McDonnell  Douglas  to  replace  16  quality  assurance  persons  who 
were  lost  to  attrition  during  1992  through  1994  even  though  the  number  of 
contracts  remained  about  the  same  at  McDonnell  Douglas. 

DCMD  Staffing  Methodologies.  The  DCMD  commanders  developed  their 
own  methodologies  for  determining  the  appropriate  staffing  level  for  each 
DCMAO  and  DPRO.  Essentially,  the  DCMDs  based  staffing  on  perceived 
workload  in  the  form  of  anticipated  contracts  assigned  to  each  DPRO  and 
DCMAO  and  other  factors  such  as  criticality  of  Ae  anticipated  work.  In 
addition,  the  DCMD  commanders  conducted  staffing  reviews  at  some  of  the 
DPROs  and  DCMAOs  each  year.  The  nature  of  these  reviews  varied  in  each 
district. 

0  DCMD  West  conducted  Command  Oversight,  Assistance,  and 
Resource  Reviews. 

o  DCMD  South  developed  a  program  entitled  "METRICS  2000"  for  the 
resource  budgeting  of  DCMAOs. 

o  DCMD  Northeast  conducted  Corporate  Business  Reviews. 

A  description  of  the  methodologies  each  DCMD  developed  is  in  Appendix  E. 
The  DCMD  staffing  methodologies  were  inadequate  because  the  meAodologies 
were  based  on  historical  information  and  routinely  included  the  previous  year's 
staffing  levels  as  a  valid  baseline  and  adjusted  the  baseline  numbers  downward. 
Staffing  estimates  were  not  based  on  a  detailed  analysis  of  personnel  needed  to 
accomplish  mission-essential  work.  DCMC  needs  to  use  a  standard 
methodology  to  determine  the  number  of  quality  assurance  specialists  required 
to  perform  documented  estimates  of  workload. 

For  example,  as  the  number  and  dollar  value  of  contracts  declined  for  Defense 
contracts  in  the  DCMD  West  region,  DCMD  West  management  determined  that 
103  quality  assurance  personnel  were  surplus.  However,  the  103  quality 
assurance  specialists  continued  on  in  their  assignments  because  DCMC  did  not 
choose  to  eliminate  their  positions  with  a  reduction  in  force.  DCMC  recognized 
a  reduction  in  force  could  adversely  effect  quality  assurance  specialists  on 
priority  programs,  and  the  surplus  quality  assurance  specialists  needed  time  to 
find  another  job.  The  method  for  determining  surplus  quality  assurance 
personnel  was  not  defined,  but  we  determined  that  the  method  used  was  not 
based  on  an  analysis  of  process-oriented  quality  assurance  workload 
requirements. 

Congressional  Concerns  with  DCMC  Staffing.  Members  of  the  House 
Committee  on  Appropriations  were  concerned  that  DCMC  lacked  a  basis  for 
determining  staffing  necessary  for  the  DCMC  civilian  contract  administrative 
workforce.  In  June  1994,  the  U.S.  House  of  Representatives,  in  Report 
No.  103-562,  "Report  of  the  Committee  on  Appropriations,  for  the 
DoD  Appropriations  Bill,  1995,"  June  27,  1994,  expressed  concerns  about  the 
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DCMC  FY  1995  budget.  Conunittee  members  agreed  that  periodic  surveys 
must  be  conducted  to  determine  the  appropriate  staffing  for  contract 
administrative  functions  within  DCMC.  DCMC  lacked  an  adequate  basis  for 
determining  staffing  of  contract  administrative  functions,  and  the  corresponding 
budget  request,  led  the  House  Committee  on  Appropriations  to  recommend  a 
$400  million  reduction  from  the  DCMC  FY  1995  budget.  On  September  29, 
1994,  the  U.S.  House  of  Representatives  issued  Report  No.  103-747,  a 
conference  report  on  appropriations  for  FY  1995,  and  agreed  to  revise  its 
original  $400  million  reduction  down  to  $36.5  million,  as  recommended  by  the 
U.S.  Senate.  In  doing  so,  the  House  Committee  on  Appropriations  stated  that 
the  $400  million  reduction  would  adversely  affect  DCMC  operations. 


Conclusion 


DCMC  managers  could  not  determine  the  size  of  the  quality  assurance  staff 
needed  to  effectively  perform  contract  quality  assurance.  DPROs  and  quality 
assurance  residencies  lacked  standard  workload  requirements  to  establish  the 
basis  for  resource  requirements. 

The  types  and  amounts  of  quality  assurance  actions  for  all  assigned  contracts 
would  provide  a  more  accurate  basis  for  the  amount  of  funds  expended  for 
quality  assurance  in  DCMC.  DCMC  quality  assurance  personnel  did  not 
review,  analyze,  document,  or  determine  the  quality  assurance  actions  necessary 
for  assigned  contracts.  Additionally,  the  implementation  of  IQUE  and 
PROCAS  did  not  have  a  high  priority  for  the  quality  assurance  personnel  in  our 
audit.  Other  than  imposed  mandatory  inspections,  quality  assurance  specialists 
have  a  wide  latitude  in  determining  Aeir  quality  assurance  priorities,  including 
the  implementation  of  IQUE  and  PROCAS.  Based  on  Finding  A,  quality 
assurance  work,  such  as  identifying  and  proofing  of  contractor  processes,  was 
generally  not  performed.  After  reviewing  the  quality  assurance  workload  at 
DPROs  and  DCMAOs,  DCMC  should  determine  whether  it  has  enough  quality 
assurance  specialists  to  fully  perform  the  quality  assurance  mission.  DCMC 
needs  to  perform  a  workload  analysis  and  an  assessment  of  manpower  needs  to 
adequately  baseline  its  quality  assurance  specialist  staffing  requirements  and  to 
enable  DCMC  to  inform  its  customers  when  DCMC  caimot  adequately  ensure 
that  DCMC  is  accepting  a  conforming  product. 


Recommendations,  Management  Comments,  and 
Audit  Response 


Revised  Recommendation.  We  revised  draft  Recommendations  2.,  3.,  and  7. 
We  deleted  the  requirement  in  draft  Recommendations  2.  and  3.  to  report 
workload  estimates  through  the  Quality  Assurance  Management  Information 
System  because  DLA  cancelled  its  4-year  development  of  die  In-Plant  Quality 
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Assurance  Records  System.  We  added  a  requirement  to  draft 
Recommendation  7.  to  establish  a  system  for  informing  the  Commander, 
DCMC,  of  the  specific  contracts  that  quality  assurance  specialists  either  would 
not  be  able  to  or  were  not  able  to  support  with  an  adequate  level  of  quality 
assurance. 

We  recommend  that  the  Director,  Defense  Logistics  Agency: 

1.  Establish  a  goal  for  developing  quality  assurance  manpower  estimates 
based  on  assessed  workload  by  FY  1996. 

DLA  Comments.  DLA  concurred  with  the  recommendation,  stating  that 
manpower  estimates  will  be  a  part  of  the  DCMC  manpower  management 
program. 

2.  Require  that  quality  assurance  specialists  document  the  estimated 
quality  assurance  workload  tasks  required  for  each  contract  and  the  time 
needed  to  proof  applicable  processes,  fully  perform  mandatory  inspections, 
product  audits,  material  reviews,  investigations  of  Product  Quality 
Deficiency  Report  and  subcontract  quality  assurance. 

DLA  Comments.  DLA  nonconcurred  with  the  specifics  of  draft 
Recommendation  2.  to  document  the  estimated  workload  related  to  material 
reviews  and  Product  Quality  Deficiency  Reports.  DLA  stated  that  it  was 
unrealistic  to  expect  finite  manpower  estimates  for  such  activities  as  material 
reviews  and  investigations  of  Product  Quality  Deficiency  Reports  before 
contract  performance.  DLA  concurred  that  the  need  for  estimated  manpower 
should  be  documented  at  each  contractor  location.  DCMC  intends  to  maintain 
management  visibility  through  a  manpower  management  program  that  is 
scheduled  for  completion  December  31,  1995. 

Audit  Response.  Our  recommendation  is  basic  to  workload  planning  and 
staffing.  The  workload  associated  with  material  reviews  and  Product  Quality 
Deficiency  Reports  is  usually  a  small  part  of  the  quality  assurance  workload  and 
can  easily  be  estimated  based  on  recent  history  at  a  contractor's  plant.  The  real 
problem  is  associated  with  the  actual  workload  estimates  based  on  current 
contracts.  The  DLA  method  does  not  measure  actual  work  requirements  and 
allows  for  inefficient  and  ineffective  quality  assurance  staffing.  Therefore,  we 
revised  our  recommendation  and  deleted  the  requirement  to  report  the  workload 
estimates  through  the  Quality  Assurance  Management  Information  System.  We 
request  that  DLA  provide  additional  comments  to  the  final  report  describing 
how  it  plans  to  measure  the  specific  work  that  quality  assurance  specialists  plan 
to  perform. 

3.  Require  that  quality  assurance  specialist  supervisors  review  and  certify 
the  process-oriented  quality  assurance  actual  workload  estimates. 

DLA  Comments.  DLA  nonconcurred  with  the  specificity  of  the  draft  report 
recommendation  to  require  supervisors  to  review  workload  estimates  as  reported 
through  the  Quality  Assurance  Management  Information  System  but  concurred 


30 


Finding  B.  Quality  Assurance  Staffing 


with  management  verification  of  workload  estimates  through  supervisory  visits 
and  management  reviews.  Also  the  DCMC  manpower  management  program 
will  also  include  methods  for  verifying  the  accuracy  of  workload  estimates. 

Audit  Response.  Although  DLA  nonconcurred  with  the  draft  report 
recommendation  to  review  workload  estimates  through  the  Quality  Assurance 
Management  Information  System,  we  consider  the  DLA  comments  responsive. 
We  deleted  the  requirement  to  review  workload  estimates  reported  through  the 
management  information  system  because  DLA  cancelled  the  In-pl^t  Quality 
Assurance  Records  system.  Therefore,  no  further  comments  are  required. 

4.  Use  documented  process-oriented  quality  assurance  workload  estimates 
as  the  basis  for  budget  requests  for  each  quality  assurance  resource. 

DLA  Comments.  DLA  concurred  with  the  recommendation  and  stated  that  it 
was  developing  a  manpower  management  program  that  would  use  appropriate 
factors  to  determine  quality  assurance  workload  estimates  at  the  contractor  plant 
level  and  that  the  action  was  scheduled  for  completion  by  December  31,  1995. 

5.  Develop  standard  methodology  to  determine  the  number  of  quality 
assurance  specialists  needed  to  perform  the  contract  quality  assurance 
mission  based  on  documented  estimates  of  workload. 

DLA  Comments.  DLA  concurred  with  the  recommendation  and  stated  that  it 
was  developing  a  manpower  management  program  that  was  scheduled  for 
completion  on  December  31,  1995. 

6.  Standardize  the  approach  and  the  rationale  for  determining  and 
describing  surplus  quality  assurance  positions  and  for  identifying  locations 
with  urgent  quality  assurance  needs. 

DLA  Comments.  DLA  concurred  with  the  recommendation  and  stated  that  it 
was  developing  a  manpower  management  program  that  was  scheduled  for 
completion  on  December  31,  1995. 

7.  Develop  a  system  for  identifying  specific  products  on  specific  contracts 
that  either  can  not  or  could  not  be  supported  with  an  adequate  level  of 
quality  assurance  so  that  the  Commander  of  the  Defense  Contract 
Management  Command  can  inform  Defense  Logistics  Agency  customers 
when  quality  assurance  specialists  either  will  not  be  able  or  were  not  able  to 
provide  an  adequate  level  of  quality  assurance  on  specific  source  inspected 
contracts. 

DLA  Comments.  DLA  concurred  with  the  recommendation  in  the  draft  report, 
stating  that  the  DCMC  policy  to  meet  its  customers  expectations  and  only  the 
Commander,  DCMC,  is  authorized  to  say  no  to  a  customer  request  for  quality 
assurance  support. 

Audit  Response.  Although  DLA  concurred  with  the  draft  report 
recommendation,  we  did  not  consider  the  DLA  comments  responsive.  During 
the  audit,  we  interviewed  quality  assurance  specialists  who  told  us  that  they 
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were  asked  to  accept  products  that  were  not  included  in  any  quality  assurance 
reviews.  The  quality  assurance  specialist  statements  were  corroborated  with  an 
absence  of  documentation  regarding  process-oriented  quality  assurance.  The 
mere  presence  of  quality  assurance  specialists  at  a  contractor's  plant  does  not 
constitute  an  adequate  level  of  quality  assurance.  Based  on  the  DLA  reply,  we 
have  revised  our  recommendation  to  create  a  system  for  informing  the 
Commander,  DCMC,  that  quality  assurance  specialists  either  were  not  able  to 
or  would  not  be  able  to  support  specific  contracts  with  adequate  quality 
assurance.  Accordingly,  we  request  t^t  DLA  provide  additional  comments  to 
reflect  how  it  intends  to  implement  a  system  for  informing  the  Commander, 
DCMC,  and,  ultimately,  the  DLA  customer  that  specific  products  either  were 
accepted  and  were  not  supported  with  an  adequate  level  of  quality  assurance,  or 
that  specific  products  are  scheduled  for  acceptance  and  DCMC  is  not  able  to 
provide  an  adequate  level  of  quality  assurance. 


Management  Comments  Required 


Management  is  requested  to  comment  on  the  items  indicated  with  an  X  in  the 
following  table. 


Table  2.  DLA  Comments  Required  on 
Findings  A  and  B  Recommendations 


Number 

Concur/ 

Nonconcur 

Proposed 

Action 

Completion 

Date 

Internal 

Control 

Issues 

A.2. 

X 

X 

X 

X 

A.3. 

X 

X 

X 

X 

A.4. 

X 

X 

X 

X 

A.5. 

X 

X 

X 

X 

B.2. 

X 

X 

X 

X 

B.7. 

X 

X 

X 

X 
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Critical  Process.  DLA  has  not  defined  critical  process.  A  similar  term, 
critical  application,  as  applied  to  weapon  system  application  items,  was  defined 
in  DLA  Regulation  3200.1  (a  Joint  Service  Regulation),  "Engineering  Support 
for  Items  Supplied  By  Defense  Logistics  Agency  and  General  Services 
Administration,"  March  1986.  A  critical  application  item  is  defined  as  an 
item  that  is  essential  to  weapon  system  performance,  the  preservation  of  life,  or 
safety  of  operating  personnel. 

Nonconforming  Material  Review.  Material  reviews  are  described  in 
DLAM  8200.5.  Material  review  procedures  exist  to  review,  evaluate,  and 
dispose  of  minor  nonconforming  supplies  or  services  before  acceptance  by  the 
Government.  As  part  of  the  material  review  process,  the  contractor  may 
establish  a  Material  Review  Board.  The  Material  Review  Board  consists  of 
technically  qualified  and  knowledgeable  contractor  personnel  and  may  include 
Government  quality  assurance  personnel.  The  Material  Review  Board  is 
responsible  for  investigating,  identifying  causes,  evaluating  performance, 
performing  trend  and  recurrence  analysis,  and  submitting  recommended 
dispositions  of  "use  as  is"  or  "repair"  of  nonconforming  material  to  Government 
quality  assurance  personnel  for  approval. 

I*rocess.  Processes  are  described  in  DLAM  8000.5,  "One  Book,"  April  3, 
1995,  as  a  series  of  tasks  leading  to  a  common  objective  and  satisfying  a 
requirement,  such  as  producing  an  estimate  or  making  a  handle.  The  processes 
described  in  DLAM  8200.5  consist  of  process  inputs  such  as  people,  materials, 
environment,  methods,  and  equipment.  Processes  can  be  related  to  product 
design,  development,  production,  and  support  and  are  intended  to  provide  a 
description  of  how  to  make  a  product. 

Product  Audits.  Product  audits,  as  described  in  DLAM  8200.5,  are 
examinations  or  tests  performed  by  Government  Quality  Assurance  Specialists 
of  contractor  produced  products  after  the  contractor  has  performed  examinations 
or  tests.  Product  audits  assess  the  contractor's  ability  to  measure  the  process 
effectively  by  examining  or  testing  products  that  the  contractor  has  previously 
determined  to  be  conforming.  Product  audits  should  confirm  that  the  contractor 
is  adequately  detecting  and  rejecting  defective  products.  Product  audits  differ 
from  process  proofings  in  that  product  audits  are  not  necessarily  conducted  at 
each  stage  in  a  process.  Product  audits  usually  assess  the  overall  effectiveness 
of  contractor  measurements  techniques  as  a  product  moves  from  one  acceptance 
point  into  the  next  manufacturing  process. 

Process  Proofing.  Process  proofing  is  described  in  DLAM  8200.5.  Process 
proofing  is  performed  to  determine  the  adequacy  of  contractor  processes 
through  thorough  review  of  inputs  employed  to  achieve  desired  outcomes. 
Process  proofing  includes  the  identification  and  flow  charting  of  the  sequence  of 
processes,  and  the  review  of  process  inputs.  The  prioritization  of  which 
processes  to  proof  first  is  determined  through  the  criticality  of  the  process. 
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Product  Quality  Deficiency  Report  Investigations,  Product  Quality 

Deficiency  Report  investigations  are  described  in  Joint  Service  Regulation 
Defense  Logistics  Agency  Regulation  (DLAR)  4155.24,  "Product  Quality 
Deficiency  Report  Program"  (the  Program).  The  Program  provides  for  the 
initial  reporting,  cause,  correction,  and  status  accounting  of  individual  product 
quality  deficiencies.  Additionally,  the  Program  data  on  product  deficiencies  are 
used  to  identify  problems,  trends,  and  recurring  deficiencies.  The  quality 
assurance  specialist  has  the  primary  responsibility  to  investigate  Product  Quality 
Deficiency  Reports  to  determine  the  cause  of  the  reported  deficiency  and 
determine  whether  the  contractor  has  taken  the  necessary  action  to  prevent  the 
same  deficiency  from  occurring  in  the  future. 

Reproofing  Processes.  Reproofing  processes  as  described  in  DLAM  8200.5 
should  take  place  when  significant  changes  occur  to  process  inputs,  for 
example,  number  of  people,  changes  in  the  type  of  equipment,  composition  of 
materials,  manufacturing  methods,  and  environmental  conditions.  Quality 
assurance  specialists  should  reproof  the  processes  affected  by  the  significant 
changes. 
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New  Hampshire 

Under  water  cable  319 

Subtotal  585 
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DCMAO  Orlando,  Florida 
Litton  Laser,  Apopka,  Florida 
Low-altitude  navigation  targeting 
infra-red  for  night 


Number  of  Critical 

_ Total  Number  of  Manufacturing  Processes _  Manufacturing  Processes 

Proofed  Since  Proofed  Between  Reproofed  Since 

Location  and  System  Id^tified  Proofed  January  L  1993  1989  and  1992  January  L  1993  Identified  Proofed 
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Appendix  D.  Summary  of  Prior  Audits  and 

Other  Reviews 


Inspector  General,  DoD 


Report  No.  93-091,  "Management  of  the  DoD  Action  Plan  for  Improving  the 
Quality  of  Spare  and  Repair  Parts,"  April  28,  1993.  The  report  states  that 
Office  of  the  Secretary  of  Defense  officials  did  not  manage  the  DoD  Action 
Plan.  The  Under  Secretary  of  Defense  for  Acquisition  and  Technology  did  not 
assign  management  of  the  DoD  Action  Plan  to  the  appropriate  action  office,  did 
not  monitor  implementation  of  the  DoD  Action  Plan,  and  did  not  change  the 
DoD  Action  Plan  as  needed.  The  report  recommended  that  the  Under  Secretary 
of  Defense  for  Acquisition  and  Tectaology  revise  and  reissue  the  DoD  Action 
Plan  every  2  years,  establish  a  feedback  system  to  monitor  DoD  Component 
implementation  of  the  DoD  Action  Plan,  assign  management  responsibility  to 
the  appropriate  office  and  update  the  March  1990  version  of  the  DoD  Action 
Plan.  The  Principal  Deputy  Assistant  Secretary  of  Defense  (Production  and 
Logistics)  concurred  with  the  intent  of  the  recommendations.  He  included 
elements  of  the  action  plan  in  DoD  4140. 1-R,  "Material  Management 
Regulation,"  and  included  a  requirement  for  the  DoD  Components  to  develop 
action  plans  that  included  performance  measures  and  milestones  in  applicable 
acquisition  phases  and  document  actions  and  accomplishments  that  implement 
quality  programs  objectives. 

Report  No.  92-099,  "Quality  Assurance  Actions  Resulting  from  Electronic 
Component  Screening,"  June  8,  1992.  The  report  describes  problems  with  the 
collection,  distribution,  and  use  of  quality  deficiency  information  in  DoD.  The 
report  also  stated  that  testing  of  electronic  components  was  inadequate  to 
identify  and  follow  up  on  contractors  who  provided  defective  electronic 
components.  In  addition,  DoD  did  not  have  effective  remedies  to  obtain 
reimbursement  or  replacement  for  major  and  critical  products  with  patent 
defects.  The  Army,  Navy,  Air  Force,  and  DLA  generally  concurred  with  the 
report's  findings  and  recommendations.  As  a  result  of  the  recommendation,  the 
Director,  Defense  Procurement,  requested  and  DLA  officials  agreed  to  identify 
problem  products  and  product  lines/suppliers  and  to  describe  ongoing,  planned, 
or  proposed  initiatives  to  address  nonconforming  products  and  possible  policy 
proposals  covering  recoupments  for  products  with  major  nonconformances. 

Report  No.  90-113,  "Nonconforming  Products  Procured  by  the  Defense 
Industrial  Supply  Center,"  September 27,  1990.  The  report  states  that,  of 
1.3  billion  parts,  27  percent  (valued  at  about  $171  million)  were  major 
nonconforming  products.  The  audit  recommendations  involved  standardizing 
definitions  for  nonconformances,  improving  new  receipt  quality  assurance 
testing,  establishing  criticality  of  spare  parts,  and  improving  the  quality 
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assurance  feedback  system.  The  DLA  implementation  of  the  DoD  Action  Plan 
for  Improving  the  Quality  of  Spare  and  Repair  Parts  satisfied  the  intent  of  the 
recommendations . 

Inspection  Report  No.  90-INS-17,  "DoD  Quality  Assurance  Program," 
August  29,  1990.  The  report  states  that  administrative  contracting  officers  were 
not  seeking  consideration  for  excessive  amounts  of  minor  nonconforming 
material.  The  report  recommended  that  DLA  establish  and  implement  policy 
that  ensured  consideration  would  be  sought  for  each  contract  containing 
nonconforming  material.  DLA  nonconcurred  with  the  recommendation,  stating 
that  DLA  policy  was  consistent  with  the  Federal  Acquisition  Regulation.  DLA 
and  IG,  DoD,  agreed  that  the  proposed  actions  in  the  DLA  implementation  of 
the  DoD  Action  Plan  for  Improving  the  Quality  of  Spare  and  Repair  Parts 
would  provide  the  needed  improvements  to  the  quality  of  products. 
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Information  Systems 


DCMD  Methodologies.  DCMC  provided  the  authority  for  determining  staffing 
levels  to  the  DCMD  commanders.  The  district  commanders  developed  separate 
and  unique  methodologies  for  determining  appropriate  staffing  for  DPROs  and 
DCMAOs.  Each  me&odology  started  with  the  past  year's  staffing  plans  and 
at^usted  the  staffing  levels  based  on  the  funding  allocations  from  DCMC. 
District  commanders  made  adjustments  on  the  basis  of  resource  reviews  of 
various  metrics  that  always  included  business  activity  in  the  form  of  contracts. 
The  adjustments  were  always  constrained  by  the  policies  and  procedures 
described  in  the  DoD  Priority  Placement  Program,  which  describes  the 
restrictions  for  reassignments  outside  of  commuting  areas.  We  analyzed  the 
staffing  methodologies  for  DCMDs  Northeast,  South,  and  West  to  determine 
how  staffing  for  quality  assurance  was  done. 

DCMD  Northeast.  DCMD  Northeast  uses  a  combination  of  workload 
measurements  and  on-site  resource  reviews  to  determine  staffing  levels.  For 
FY  1994,  21  corporate  business  reviews  were  performed  to  relate  the  previous 
years  staffing  and  workload  with  the  estimates  for  future  work  and  the  staffing 
needed  to  satisfy  the  estimated  work  load.  In  addition,  each  DPRO  and 
DCMAO  commander  conducted  a  staffing  self  assessment.  All  assessments 
were  baselined  on  September  1993  staffing  levels  and  were  evaluated  against 
various  factors  including: 

o  projected  volume  of  Defense  contracts, 

0  percentage  of  completion  of  existing  contracts, 

o  reductions  in  contractor  workforce, 

0  number  of  large  contracts, 

o  geographic  dispersion  of  contractor  sites, 

o  projected  reorganization  or  realignment, 

o  unusual  customer  inspection  requirements,  and 

o  prior  staffing  adjustments. 


The  combined  corporate  business  reviews  for  DCMD  Northeast  recommended 
an  overall  reduction  of  280  contract  administration  personnel  from  the 
1993  baseline.  Although  the  reviews  considered  the  level  of  quality  assurance 
required  through  review  of  historical  workload  indicators,  the  number  of  quality 
assurance  specialists  reconunended  was  not  broken  out.  The  level  of  staffing 
for  quality  assurance  was  left  to  the  individual  DCMAO  and  DPRO  commander 
within  the  commander's  overall  authorized  staffing  limits. 
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DCMD  West.  DCMD  West  relies  primarily  on  Command  Oversight, 
Assistance,  and  Resource  Reviews  (Command  Reviews)  and  headquarters-level 
evaluations  of  projected  workload  to  determine  staffing  levels.  Command 
Reviews  are  assessments  of  operational  support  functions  to  determine  whether 
DPROs  and  DCMAOs  have  implemented  DLA  regulations  and  policies. 
Command  Reviews  also  evaluate  DPRO  and  DCMAO  staffing  needs. 

DCMD  West  scheduled  Command  Reviews  for  19  DCMAOs  and  DPROs  from 
October  1993  through  December  1994.  Of  the  19  Command  Reviews  visits 
scheduled,  8  reports  were  completed  before  the  end  of  our  audit  in 
August  1994.  The  Command  Reviews  as  of  August  31,  1994,  recommended  a 
reduction  of  50  personnel  at  7  sites  and  an  increase  of  15  at  2  sites.  The  DPRO 
and  DCMAO  coitunanders  would  determine  at  which  sites  to  reduce  personnel. 
The  Command  Reviews  evaluated  historical  workload,  including 

0  the  number  of  open  contracts, 

0  the  complexity  of  items  being  manufactured, 

o  the  number  of  mandatory  inspections, 

0  administrative  support  activities, 

0  material  review  board  activity,  and 

o  corrective  action  requests. 

In  their  review  of  staffing  levels  and  workload,  the  Conunand  Reviews 
identified  quality  assurance  specialists  performing  nonvalue  added  tasks,  or 
having  little  work,  and  determined  that  the  lack  of  a  management  data  system  to 
fully  assess  workload  obscured  any  further  visibility  of  resource  allocations. 
We  interviewed  members  of  the  Command  Review  teams  and  were  told  that  the 
resource  portion  of  the  review  lacked  guidance,  structure,  consistency,  and 
objectivity  for  determining  appropriate  staffing  levels.  Although  the  Command 
Review  team  reports  stated  llieir  staffing  level  recommendations  were  based  on 
workload,  the  recommendations  were  based  on  estimates  from  workload  factors. 
There  were  no  indications  that  the  quality  assurance  activities  required  for  the 
contracts  assigned  to  each  DPRO  was  evaluated  to  provide  a  basis  for  future 
staffing  levels. 

DCMD  South.  DCMD  South  uses  two  systems  to  determine  staffing  needs.  A 
statistical  model  called  METRICS  20(K)  is  used  for  allocating  funds  to  the 
DCMAOs,  while  a  less  scientific  approach  for  equitable  allocation  based  on 
contractor  environment,  contract  workload  volume,  and  other  unique  factors 
was  used  for  the  DPROs. 

The  METRICS  2000  model  applies  regression  analysis  to  performance 
indicators.  The  quality  assurance  indicator  used  to  determine  adequate  staffing 
levels  is  the  number  of  contracts  on-hand  adjusted  by  a  factor  for  the  level  of 
quality  oversight  required.  The  three  categories  for  weighted  factors  are 
MIL-Q-9858A  contracts,  MIL-I-45208  contracts,  and  all  other  contracts. 
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MIL-Q  is  graded  as  a  higher  level  of  quality  assurance  standard  and  is 
considered  to  require  a  greater  level  of  effort  over  a  longer  duration. 
METRICS  2000  does  not  attempt  to  project  the  work  needed  to  satisfy  process 
oriented  quality  assurance. 

A  separate  model  was  developed  for  the  10  largest  DPROs  because  the 
performance  indicators  do  not  provide  a  valid  correlation  for  smaller  DPROs. 
The  DPRO  model  applies  regression  analysis  to  three  overall  indicators: 

0  total  active  contracts, 

o  unliquidated  obligations,  and 

0  the  number  of  contractor  personnel  working  on  Government  contracts. 

Both  the  DCMAO  and  the  DPRO  models  rely  on  information  from  the  Quality 
Assurance  Management  Information  System.  Though  the  models  contained 
adjustment  factors  for  level  of  effort  for  quality  assurance  requirements,  the 
METRICS  2000  models  were  designed  to  determine  aggregate  staffing  levels 
based  on  the  volume  of  Government  business  rather  than  the  number  of  work 
hours  required  to  accomplish  directed  quality  assurance  tasks. 

Conclusion.  The  DCMDs  use  historical  information  as  the  bases  for  the 
staffing  methodologies.  DCMDs  do  not  use  projected  workload  derived  from 
an  analysis  of  the  tasks  required  to  adequately  accomplish  process-oriented 
quality  assurance. 


DLA  Unit  Cost 


DLA  relies  on  historical  costs  to  determine  the  basis  for  future  funding  of 
DCMC  staffing.  The  adequacy  of  staffing  is  not  fully  considered  in  the 
analyses  of  the  unit  cost  information  collected. 

Unit  Cost  System.  DCMC  used  information  from  the  DLA  Unit  Cost  System 
for  budget  planning  and  resource  allocation.  Unit  cost  provides  the  acmal  cost 
of  producing  18  product  and  service  categories  DCMC- wide.  The  products  and 
services  were  accounted  for  without  regard  to  organizational  strucmre.  DoD 
and  DLA  formulated  the  DCMC  budget  based  on  the  combined  data  for  all 
18  unit  cost  codes  and  predicted  work  volume.  DCMC  then  allocated  each 
DCMD  budget  based  on  unit  cost  data  and  the  overall  amount  provided  from 
DLA. 

DCMC  separated  labor  hours  and  costs  into  18  products  and  services  to  predict 
work  volume  for  the  subsequent  budget  year.  Unit  cost  codes  10,  "^ality 
Assurance,"  and  11,  "Mandatory  QA  Inspection  Requirements,"  are  the 
two  primary  codes  associated  with  quality  assurance. 
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0  Unit  cost  code  10,  "Quality  Assurance,"  is  based  on  the  number  of 
contracts  closed. 

o  Unit  cost  code  11,  "Mandatory  QA  Inspection  Requirements,"  is 
based  on  the  number  of  quality  assurance  letters  of  instruction  received. 

The  letters  of  instruction  provide  details  on  the  specific  mandatory  quality 
assurance  inspections  to  be  performed. 

DCMC  uses  the  unit  cost  method  to  determine  the  annual  budget  for  the 
DCMDs.  Hie  unit  cost  method  uses  the  number  of  contracts  closed  as  the  basis 
for  determining  cost  per  unit  of  administering  contract  quality  assurance. 
Unfortunately,  the  number  of  contracts  closed  does  not  effectively  consider 
quality  assurance  workload  because  the  amount  of  quality  assurance  work  varies 
between  contracts.  For  example,  the  quality  assurance  work  on  1  contract  could 
exceed  the  cumulative  quality  assurance  work  on  100  contracts. 

Activity-Based  Costing.  Activity-based  costing  is  an  analytical  methodology 
that  assesses  the  cost  of  the  day-to-day  operating  processes  prescribed  by 
DCMC  policy  directives.  Unit  cost  separates  labor  hours  and  costs  into  18  unit 
cost  codes  covering  all  areas  of  contract  administration,  including  quality 
assurance.  Activity-based  costing  further  identifies  117  separate  activities 
within  each  of  the  18  unit  cost  codes.  The  117  activity-based  costing  activities 
are  intended  to  identify  for  what  purpose  resources  are  expended  in  die  18  unit 
cost  codes.  According  to  DCMC  officials,  however,  activity-based  costing  is 
based  on  historical  data,  only  identifying  the  purposes  for  which  resources  were 
expended,  and  is  not  a  method  to  determine  the  adequacy  of  quality  assurance 
resources  necessary  for  operational  effectiveness. 
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Appendix  F.  Summary  of  Potential  Benefits 

Resulting  From  Audit 

Recommendation 

Reference  Description  of  Benefit  Type  of  Benefit 

A.l.  Program  Results.  Standardizes  a  Nonmonetary, 

key  concept  of  PROCAS  in 
DLAM  8000.5  by  defining  the 
term,  "critical  manufacturing 
process. " 

A. 2. a.  Internal  Controls.  Establishes  Nonmonetary, 

accountability  for  verifying 
compliance  with  FAR 
documentation  requirements  to 
document  the  scope  of  quality 
assurance  work  required. 

A.2.b.  Internal  Controls.  Establishes  Nonmonetary, 

accountability  for  accurately 
identifying  critical  quality  assurance 
work. 

A.2.C.  Internal  Controls.  Establishes  Nonmonetary, 

accountability  for  requiring  full 
.  documentation  of  process  proofing 
of  contractor  process  inputs. 

A.2.d.  Internal  Controls.  Establishes  Nonmonetary, 

accountability  for  requiring  full 
documentation  of  reproofing 
contractor  processes  when  changes 
occur.  Establishes  requirement  to 
certify  every  2  years  when  changes 
have  not  occurred. 

A.2.e.  Internal  Controls.  Establishes  Nonmonetary, 

accountability  for  requiring  the 
reporting  of  processes  identified, 
critical  priority  processes  identified, 
and  processes  either  proofed  or 
reproofed  through  the  Quality 
Assurance  Information  Management 
System. 


Appendix  F.  Sununary  of  Potential  Benefits  Resulting  From  Audit 


Recommendation 

Reference 


A.3.a. 


A.3.b. 


A.3.C. 


A.4. 


A.S.a. 


A.5.b. 


B.l. 


Description  of  BeneHt 


Internal  Controls.  Establishes 
accountability  for  requiring  quality 
assurancce  specialist  supervisors  to 
review  and  evaluate  the  adequacy  of 
processes  identified. 

Internal  Controls.  Establishes 
accountability  for  requiring  quality 
assurance  specialist  supervisors  to 
review  and  evaluate  the  adequacy  of 
documentation  for  proofed  and 
reproofed  processes. 

Internal  Control.  Establishes 
accountability  for  requiring  quality 
assurance  specialist  supervisors  to 
review  and  evaluate  the  accuracy  of 
information  reported  through  the 
Quality  Assurance  Management 
Information  System. 

Internal  Controls.  Establishes  an 
initiative  to  measure  the 
effectiveness  of  each  contractor's 
quality  control  program. 

Internal  Controls.  Establishes  a 
requirement  in  the  internal 
Management  Control  Program  to 
minimize  quality  assurance  risks. 

Internal  Controls.  Establishes 
quality  assurance  as  a  high  risk  area 
in  the  internal  management  control 
program. 

Program  Results.  Establishes  a 
DLA  goal  for  basing  quality 
assurance  manpower  on  workload 
assessments. 


Type  of  Benefit 


Nonmonetary. 


Nonmonetary. 


Nomnonetary. 


Nonmonetary. 


Nonmonetary. 


Nonmonetary. 


Nonmonetary. 
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Recommendation 

Reference  Description  of  Beneflt  Type  of  Benefit 


B.2. 

Internal  Controls.  Requires 
documenting  of  quality  assurance 
workload  tasks  and  reporting 
through  the  Quality  Assurance 
Management  Information  System. 

Nonmonetary. 

B.3. 

Internal  Controls.  Requires 
supervisory  review  and  validation  of 
quality  assurance  workload  data 
input  to  the  Quality  Assurance 
Management  Information  System. 

Nonmonetary. 

B.4. 

Internal  Controls.  Requires 
documentation  of  worldoad  tasks  to 
establish  the  baseline  for  staffing 
decisions. 

Nonmonetary. 

B.5. 

Program  Results.  Establishes  a 
standard  methodology  basis  for 
determining  staffing  levels. 

Nonmonetary. 

B.6. 

Internal  Controls.  Establishes  a 
standard  approach  for  determining 
surplus  and  needed  quality 
assurance  personnel  by  location. 

Nonmonetary. 

B.7. 

Internal  Controls.  Establishes 
policy  requiring  DCMC  to  inform 
customers  when  quality  assurance 
specialists  cannot  perform  the 
quality  assurance  required  on 
specific  source  inspected  contracts. 

Nonmonetary. 
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Appendix  G.  Organizations  Visited  Or 

Contacted 


Defense  Organizations 

Defense  Logistics  Agency,  Alexandria,  VA 

Defense  Contract  Management  Command,  Alexandria,  VA 
Defense  Contract  Management  District  Northeast,  Boston,  MA 
Defense  Plant  Representative  Offices 
General  Electric,  Lynn,  MA 
Raytheon,  Burlington,  MA 

Defense  Contract  Management  Area  Operations  Boston,  MA 
Defense  Contract  Management  Area  Operations  Philadelphia,  PA 
Defense  Contract  Management  District  South,  Atlanta,  GA 
Defense  Contract  Management  Area  Operations 
Baltimore,  MD 
Clearwater,  FL 
Orlando,  FL 
Dallas,  TX 
Atlanta,  GA 
Birmingham,  AL 

Defense  Hant  Representative  Offices 
Martin  Marietta,  Orlando,  FL 
Lockheed,  Marietta,  GA 

Defense  Contract  Management  District  West,  El  Segundo,  CA 
Defense  Contract  Management  Area  Operations 
El  Se^ndo,  CA 
San  Diego,  CA 
San  Francisco,  CA 
Van  Nuys,  CA 

Defense  Plant  Representative  Offices 
McDonnell  Douglas,  St.  Louis,  MO 
McDoimell  Douglas,  Long  Beach,  CA 
McDonnell  Douglas,  Huntington  Beach,  CA 
FMC,  San  Jose,  CA 
Hughes,  Fullerton,  CA 
Lockheed,  Sunnyvale,  CA 
TRW,  Redondo  Beach,  CA 
Northrop,  Hawthorne,  CA 
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Office  of  the  Secretary  of  Defense 

Under  Secretary  of  Defense  for  Acquisition  and  Technology 
Deputy  Under  Secretary  of  Defense  (Acquisition  Reform) 
Deputy  Under  Secretaiy  of  Defense  (Logistics) 

Director  of  Materiel  and  Resource  Management 
Under  Secretary  of  Defense  (Comptroller)  Financial  Officer- 
Assistant  Secretary  of  Defense  (Economic  Security) 

Director,  Administration  and  Management 
Program  Manager,  Total  Quality  Management 


Defense  Organizations 

Director,  Defense  Logistics  Agency 

Commander,  Defense  Contract  Management  Command 


Non-Defense  Federal  Organizations 

Office  of  Management  and  Budget 

Technical  Information  Center,  National  Security  and  International  Affairs  Division, 
General  Accounting  Office 

Chairman  and  Ranking  Minority  Member  of  Each  of  the  Following  Congressional 
Committees  and  Subcommittees: 

Senate  Conunittee  on  Appropriations 

Senate  Subcommittee  on  Defense,  Committee  on  Appropriations 
Senate  Committee  on  Armed  Services 
Senate  Committee  on  Governmental  Affairs 

Senate  Subcommittee  on  Manpower  and  Personnel,  Committee  on  Armed  Services 
House  Committee  on  Appropriations 

House  Subcommittee  on  National  Security,  Committee  on  Appropriations 
House  Committee  on  Government  Reform  and  Oversight 
House  Subcommittee  on  National  Security,  International  Affairs  and  Criminal 
Justice,  Committee  on  Government  Reform  and  Oversight 
House  Committee  on  National  Security 
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Part  IV  -  Management  Comments 


ss 


Defense  Logistics  Agency  Comments 


IN  RCPLV 

»CFE»  TO  DDAI 


DEFENSE  LOGISTICS  AGENCY 
HEADQUARTERS 
CAMERON  STATION 
ALEXANDRIA.  VIRGINIA  22304-6100 


3  MARCH  1995 


MEMORANDUM  FOR  THE  ASSISTANT  INSPECTOR  GENERAL  FOR  AUDITING, 
DEPARTMENT  OF  DEFENSE 


) 


SUBJECT:  DoD  IG  Draft  Report,  Defense  Contract  Management 
Command  Management  of  Quality  Assurance  Resources 
(Project  No.  3CF-0071) 


This  is  in  response  to  your  21  December  1994  request. 


14  Enclosures 


CC  : 

AQCBA 

AQCOG 
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TYPE  OF  REPORT:  Audit  PURPOSE  OP  INPUT:  Initial  Position 

AUDIT  TITLE;  DCMC  Management  of  Quality  Assurance  Resources 
(Project  No.  3CF-0071) 

FINDING  A;  DCMC  did  not  effectively  manage  the  quality  assurance 
workforce.  Quality  assurance  specialists  and  supervisors  at  all 
of  the  13  audit  sites  did  not  effectively  implement  the  In-Plant 
Quality  Evaluation  or  the  Process-Oriented  Contract 
Administration  Services  programs  (IQUE  and  PROCAS) .  Specifically, 
these  conditions  occurred  because  DCMC: 

did  not  adequately  prioritize  the  need  to  identify  critical 
manufacturing  processes  for  IQUE  and  PROCAS; 

did  not  hold  quality  assurance  specialists  accountable  for 
implementing  either  IQUE  or  PROCAS; 

did  not  hold  supervisors  responsible  for  reviewing  and 
evaluating  the  identification  of,  and  proofing  of,  processes; 

did  not  effectively  follow-up  on  a  1992  action  plan  to  improve 
the  IQUE  program;  and 

did  not  report  realistic  assessments  of  the  risk  associated 
with  IQUE  in  its  Internal  Management  control  Plan. 

Consequently,  neither  IQUE  nor  PROCAS  provided  adequate  assurance 
that  the  products  the  Government  accepted  were  produced  under  a 
reliable  process  that  would  consistently  produce  a  conforming 
product.  In  addition,  QASs  did  not  take  advantage  of  process- 
oriented  quality  assurance  to  establish  a  basis  for  substituting 
process  proofing  and  product  audits  for  more  labor-intensive  and 
expensive  mandatory  inspections. 

DLA  COMMENTS:  Nonconcur.  The  IG  does  not  provide  sufficient 
evidence  for  the  finding  that  DCMC  did  not  effectively  manage 
quality  assurance  specialists,  or  that  the  IQUE  and  PROCAS 
methodologies  were  not  effectively  implemented  and  managed  at  the 
13  audit  sites.  We  disagree  with  the  IG*s  position  relating  to 
what  constituted  acceptable  implementation  of  IQUE  and  PROCAS  and 
what  level  of  documentation  with  respect  to  process  proofing  is 
required  to  be  maintained.  In-plant  personnel  and  supervisors 
have  been  and  still  are  held  responsible  for  assuring  that 
contractors  provide  only  conforming  product  to  our  customers. 
Although  the  report  indicates  that  DCMC  assurance  actions  are 
inadequate,  no  evidence  concerning  defective  product  accepted  at 
the  13  audit  sites  is  presented  to  support  the  statement. 

The  adoption  of  a  process  oriented  methodology  for  contract 
administration  was  a  significant  change  in  the  way  DCMC  verified 
contractor  control.  Throughout  the  transition  to  this 
methodology,  the  Agency  has  been  identifying  and  making  changes 
to  improve  the  approach  as  necessary.  This  has  been  especially 
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true  with  respect  to  selecting  and  prioritizing  processes  for 
proofing  and  improvement.  The  IQUE  manual  did  not  specifically 
provide  prioritization  criteria  for  process  selection  and 
proofing.  Rather,  the  application  of  the  IQUE  guidelines  were  to 
be  tailored  to  specific  facilities,  product  lines,  and 
or/processes  as  judged  necessary  by  DCMC  personnel  to  implement 
an  effective  and  efficient  IQUE  program.  Based  upon  feedback  and 
experience,  prioritization  is  now  more  specifically  addressed 
and  defined  in  the  PROCAS  and  Product  and  Manufacturing  Assurance 
chapters  of  the  DCMC  "One  Book”.  The  fact  that  it  wasn't  done 
earlier  is  not,  in  our  opinion,  evidence  of  mis-management  or 
mis-application  of  policy.  Appropriate  changes  were  being  put  in 
place  prior  to  the  initiation  of  the  subject  audit.  The  process 
oriented  approach  promulgated  by  IQUE  and  PROCAS  is  site  specific 
and  relies  heavily  upon  the  professional  judgement  of  the  onsite 
QA  personnel,  using  customer  input  and  data  to  select  and 
prioritize  processes,  A  wide  variation  of  approaches  to 
verifying  contractor  control  of  product  quality  can  be  expected. 
Variation  is  not  indicative  of  mis-management  or  poor 
implementation , 

The  follow-on  recommendations  to  this  finding  offered  by  the 
audit  team  builds  a  generic  improvement  scenario.  It  addresses  a 
complete  management  accountability  process  that  encompasses 
everything  from  individual  performance  appraisals  through  the 
Agency's  internal  control  program.  However,  the  improvement 
scenario  offered  is  at  times  ill  founded  because  of  the  "critical 
process"  concept  applied  by  the  audit  team  during  the  review  and 
the  auditor's  mis-interpretation  of  what  constituted  sufficient 
and  proper  process  proofing  documentation.  These  two  factors  led 
to  an  inaccurate  assessment  of  the  overall  adequacy  of  DCMC's 
in-plant  surveillance  activities.  While  we  may  not  agree  with 
all  of  the  specifics  of  the  management  accountability  scheme 
offered  by  the  report,  we  recognize  the  benefits  of  having  such 
mechanisms  in  place  is  beneficial  and  we  are  taking  steps  to 
improve  our  actions  in  those  areas.  Specific  actions  will  be 
addressed  when  commenting  upon  each  recommendation.  While  we 
agree  there  is  room  for  improvements  in  the  manner  of  how  PROCAS 
is  defined  and  implemented,  the  audit  team’s  assertions  and 
conclusions  do  not  accurately  reflect  the  sufficiency  of  the 
Agency's  efforts  in  the  area  of  (quality  assurance. 

INTERNAL  MANAGEMENT  CONTROL  WEAKNESSES; 

(x)  Nonconcur 

ACTION  OFFICER:  Robert  Schmitt,  AQCOG 
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TYPE  OF  REPORT:  Audit  PURPOSE  OF  INPUT:  Initial  Position 

AUDIT  TITLE:  DCMC  Management  of  Quality  Assurance  Resources 
(Project  No.  3CF-0071) 

RECOMMENDATION  A.l:  Recoitimend  that  the  Director,  Defense 
Logistics  Agency,  define  in  the  Defense  Logistics  Agency  Manual 
8000.5  "One  Book"  (draft)  October  1994,  the  terra  "critical 
manufacturing  process"  so  that  the  meaning  and  intent  of  the  terra 
is  clear, 

DLA  COMMENTS;  Concur.  Clarification  of  the  terra  "critical 
process"  was  required,  and  accomplished  via  DCMC  Policy  Letter 
94-14,  "Reporting  PROCAS  Progress."  This  policy  letter  was 
drafted  prior  to  release  of  the  DoD  IG*s  initial  draft  report  and 
published  11  Oct  94.  A  process  is  considered  critical  if  during 
process  selection  it  is  found  to  significantly  affect  the  cost, 
schedule,  or  technical  performance  of  the  product  or  service. 

This  definition  will  also  be  incorporated  into  the  DCMC  "One 
Book"  chapter  on  PROCAS,  at  next  revision. 

DISPOSITION: 

(x)  Action  is  Ongoing.  Estimated  completion  Date:  30  Apr  95 

INTERNAL  MANAGEMENT  CONTROL  WEAKNESSES: 

(x)  Concur;  however  weakness  is  not  considered  material 

ACTION  OFFICER;  Robert  Schmitt,  AQCOG 

PSE  REVIEW/APPROVAL;  Robert  P.  Scott,  Exec  Dir,  Contract  Mgmt, 
COORDINATION:  Dave  Stumpf,  DDAI  ^ 

OD/OT, 

DLA  APPROVAL: 


Defense  Logistic  Agency  Comments 


Final  Report 
Reference 


Revised. 


TYPE  OF  REPORT;  Audit  PURPOSE  OF  INPUT:  Initial  Position 

AUDIT  TITLE:  DCMC  Management  of  Quality  Assurance  Resources 
{Project  No.  3CF-0071) 

RECOMMENDATION  A. 2:  Require  that  DCMC  establish  a  system  of 
accountability  in  either  Defense  Logistic  Agency  Manual  8000.5/ 
"One  Book"  or  in  Defense  Logistics  Agency  Manual  8200.5,  "In- 
Plant  Quality  Evaluation,"  and  in  standard  performance  plans  to 
make  quality  assurance  specialists  responsible  for  fully 
implementing  process-oriented  quality  assurance  by; 

a.  Identifying  a  complete  universe  of  processes  and  sub¬ 
processes  to  a  level  of  detail  that  satisfactorily  describes  the 
contractor's  sequence  of  processes  for  producing  conforming 
products  in  accordance  with  each  contract  assigned. 

b.  Identifying  critical  processes  and  documenting  why  the 
process  is  critical, 

c.  Fully  documenting  process  proofing  of  contractor  process 
inputs  to  including  equipment,  materials,  people,  methods,  and 
environmental  controls,  and  either  flowcharting  extensive  or 
complex  processes  or,  for  simple  processes,  listing  the  sequence 
of  applicable  operations. 

d.  Fully  documenting  process  reproofings  and  the  changes  in 
contractor  processes  every  2  years.  If  the  contractor  processes 
do  not  change,  document  the  fact  that  no  changes  had  occurred, 

e.  Requiring  quality  assurance  specialists  to  report  through 
the  Quality  Assurance  Management  Information  System  processes  and 
critical  processes  identified,  proofed,  and  reproofed. 

DLA  COMMENTS;  Nonconcur.  It  is  not  always  necessary  to  identify 
the  "universe  of  processes,"  or  to  require  a  proofing  of  each 
process  every  two  years.  For  example,  where  analysis  of  credible 
data  indicates  processes  are  consistently  producing  conforming 
products,  it  may  not  be  necessary  or  prudent  to  expend  DCMC 
resources  identifying  and  proofing.  We  believe  that  significant 
events  (e.g.,  Delta  change  to  single  cell)  and  data,  not 
arbitrary  minimum  schedules,  should  drive  the  frequency  and 
intensity  of  DCMC  oversight,  DCMC  policies  give  the  CAOs  the 
flexibility  to  structure  effective  programs.  PROCAS  Teams  are 
already  required  (see  DCMC  "One  Book"  Chapter  on  PROCAS,  under 
step  5,  "Understanding  the  Process")  to  document  process 
proofing,  including  the  techniques  used,  the  process  inputs  and 
outputs  proofed,  where  the  proofing  was  performed,  who  performed 
the  proofing,  proofing  results,  and  actions  taken  based  on 
proofing  results.  Furthermore,  PROCAS  Teams  are  already 
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responsible  (see  DCMC  '*One  Book"  Chapter  on  PROCAS,  under  step  4, 
"Process  Selection"  and  DCMC  Policy  Letter  94-14,  "Reporting 
PROCAS  Progress")  for  identifying  critical  processes  (if  any)  . 

The  team  leader’s  role  is  to  guide,  support,  and  manage. 
Individual  accountability  is  maintained  through  routine  team 
leader  communication  with  individual  specialists,  individual 
performance  plans,  and  individual  appraisals.  We  feel  that  this 
is  an  effective  method  for  tracking  accountability,  better  than 
the  recommended  use  of  the  Quality  Assurance  Management 
Information  System.  We  do  not  believe  it  is  necessary  to  mandate 
additional  documentation  requirements  to  maintain  accountability. 
However,  we  recognize  that  documentation  at  some  audit  sites  was 
insufficient.  We  will  issue  guidance  in  a  letter  to  reemphasize 
the  need  for  adequate  documentation  and  re-proofing  to  all  DCMC 
field  activities  and  verify  compliance  during  assessment 
activities . 


DISPOSITION: 

(X)  Action  is  Ongoing.  Estimated  completion  Date:  31  Mar  95 

INTERNAL  MANAGEMENT  CONTROL  WEAKNESSES: 

(x)  Nonconcur 


ACTION  OFFICER:  Robert  Schmitt,  AQCOG 

PSE  REVIEW/APPROVAL:  Robert  P.  Scott,  Exec  Dir,  Contract  Mgmt, 
COORDINATION:  Dave  Stumpf,  DDAI 


DLA  APPROVAL: 


Final  Report 
Reference 


Revised 
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Final  Report 
Reference 


Revised 


TYPE  OF  REPORT:  Audit  PURPOSE  OF  INPUT:  Initial  Position 

AUDIT  TITLE:  DCMC  Management  of  Quality  Assurance  Resources 
(Project  No.  3CF-0071) 

RECOMMENDATION  A.  3;  Require  that  DCMC  establish  a  system  of 
accountability  in  either  Defense  Logistics  Agency  Manual  8000.5 
"One  Book"  or  in  Defense  Logistics  Agency  Manual  8200.5/  "In- 
Plant  Quality  Evaluation,"  and  in  standard  performance  plans  to 
make  the  supervisors  of  quality  assurance  specialists  responsible 
for  reviewing  and  evaluating: 

a.  The  adequacy  of  the  universes  of  processes  identified, 

b.  The  adequacy  of  documentation  supporting  processes  proofed 
and  reproofed. 

c.  The  accuracy  of  the  information  reported  by  quality 
assurance  specialists  through  the  Quality  Assurance  Management 
Information  system. 

DLA  COMMENTS:  Nonconcur.  As  discussed  above  (under  Rec  A. 2),  we 
do  not  believe  it  is  always  necessary  to  identify  the  "universe 
of  processes"  and  supervisors  are  already  responsible  through 
performance  appraisals  for  guiding,  supporting  and  managing  the 
work  of  their  employees.  Similarly,  supervisors  are  already 
responsible  for  guiding,  supporting,  and  managing  information 
gathering  and  reporting.  That  doesn't  mean  we  are  satisfied  with 
status  quo.  The  evolving  DCMC  Quality  Improvement  Criteria  and 
Assessment  Architecture  (see  DCMC  "One  Book"  Chapter  on 
Assessment  Architecture)  provides  a  framework  in  which  the 
adequacy  of  the  approach,  deployment,  and  results  of  all 
management  controls,  information  systems  and  operational 
processes  can  be  uniformly  evaluated  and  independently  validated. 
The  criteria  and  assessment  architecture  are  akin  to  the  Malcolm 
Baldrige  National  Quality  Award  Program.  It  is  an  integrated, 
corporate  approach  to  accomplishing  and  evaluating  management 
controls  at  each  DCMC  organizational  level,  including 
Headquarters,  Districts,  and  CAOs.  The  purpose  of  these 
assessments  is  to  identify  the  health  of  operations,  and  pinpoint 
where  process  improvements  may  be  needed.  The  assessment  process 
tools  include  self-assessments,  internal  operations  assessments, 
internal  customer  support  visits,  contractor  assessments,  special 
process  reviews,  and  culminating  in  Commanders*  Annual  Statements 
of  Assurance,  The  Assessment  Structure  through  the  Unit  Self 
Assessments  will  specifically  address  the  efficiency  and 
effectiveness  of  the  process  inputs  and  outputs  in  DLAM  8000.5 
DCMC  Contract  Management,  i.e.  the  adequacy  of  the  Product  and 
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Manufacturing  Assurance  surveillance  plan  and  the  sufficiency  of 
its  implementation.  We  believe  this  approach  will  lead  DCMC  to 
significant,  long-term  improvements  in  performance,  efficiency, 
and  customer  satisfaction, 

DISPOSITION: 

(x)  Action  is  Considered  Complete. 

INTERNAL  MANAGEMENT  CONTROL  WEAKNESSES: 

(x)  Nonconcur 

ACTION  OFFICER:  Robert  Schmitt,  AQCOG 

PSE  REVIEW/APPROVAL:  Robert  P.  Scott,  Exec  Dir,  Contract  Mgmt, 
COORDINATION :  Dave  Stumpf ,  DDAI  ^ 


DLA  APPROVAL: 


0  j  Uti\?,  :9S5 
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Final  Report 
Reference 


Revised 


TYPE  OF  REPORT:  Audit  PURPOSE  OF  INPUT:  Initial  Position 

AUDIT  TITLE:  DCMC  Management  of  Quality  Assurance  Resources 
(Project  No.  3CF-0071) 

RECOMMENDATION  A. 4:  Implement  Rec  B.S.b  in  Inspector  General, 
DoD,  Report  No.  94-079,  ”DoD  Components  Implementing  Action  Plans 
for  Improving  the  Quality  of  Spare  Parts,"  April  12,  1994,  to 
revise  the  Defense  Logistics  Agency  Action  Plan  for  Continuously 
Improving  the  Quality  of  Spare  and  Repair  Parts,  adding  an 
objective  to  measure  the  extent  to  which  each  DoD  contractor's 
quality  control  processes  are  reviewed  and  validated. 

DLA  COMMENTS;  Nonconcur.  We  do  not  believe  that  merely  counting 
the  tasks  we  perform  is  a  good  measure  of  performance.  And  for 
reasons  previously  discussed  (under  Rec  A.2) ,  we  do  not  believe 
it  is  always  necessary  to  count  the  processes  in  the  "universe." 
It  is  therefore  inappropriate  to  use  that  "universe"  as  a 
baseline  against  which  process  review  and  validation  efforts  are 
measured.  As  explained  above  (under  Rec  A. 3),  our  evolving 
assessment  methodology  focuses  on  local  responsibility  and 
accountability,  using  the  process  outlined  in  the  Assessment 
Architecture  chapter  of  the  "One  Book",  It  evaluates  the 
effectiveness  of  the  local  approach,  deployment,  and  results, 
rather  than  just  compliance  with  mandated,  detailed  documentation 
requirements . 

DISPOSITION; 

(x)  Action  is  Considered  Complete. 

INTERNAL  MANAGEMENT  CONTROL  WEAKNESSES;  , 

(x)  Nonconcur 

ACTION  OFFICER:  Robert  Schmitt,  AQCOG 

PSE  REVIEW/APPROVAL;  Robert  P.  Scott,  Exec  Dir,  Contract  Mgmt, 
COORDINATION:  Dave  Stumpf,  DDAI 

DLA  APPROVAL: 


Jv**  'I  .!*■.  ’'£*»•  i  I 

Pi'huijjui  Dois-uly  Dii-cctor^ 
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TYPE  OF  REPORT:  Audit  PURPOSE  OF  INPUT;  Initial  Position 

AUDIT  TITI^E:  DCMC  Management  of  Quality  Assurance  Resources 
(Project  No.  3CF-0071) 

RECOMMENDATION  A. 5:  Establish  a  requirement  in  the  Internal 
Management  Control  Program  to  evaluate  process-oriented  quality 
assurance  in  accordance  with  criteria  that  measure  the  execution 
of  In-Plant  Quality  Evaluation  and  Process-Oriented  Contract 
Administration  services  in  terms  of  the  extent  that  each  DoD 
supplier’s  quality  control  processes  are  reviewed  and  validated. 

DLA  COMMENTS.  Nonconcur.  We  do  not  believe  that  merely  counting 
the  tasks  we  perform  is  a  good  measure  of  performance.  And  for 
reasons  previously  discussed  (under  Rec  A, 2),  we  do  not  believe 
it  is  always  necessary  to  count  the  processes  in  the  "universe." 
It  is  therefore  inappropriate  to  use  that  "universe"  as  a 
baseline  against  which  process  review  and  validation  efforts  are 
measured.  As  explained  above  (under  Rec  A. 3),  our  evolving 
assessment  methodology  focuses  on  local  responsibility  and 
accountability/  using  the  process  outlined  in  the  Assessment 
Architecture  chapter  of  the  "One  Book".  It  evaluates  the 
effectiveness  of  the  local  approach,  deployment,  and  results, 
rather  than  just  compliance  with  mandated,  detailed  documentation 
requirements , 

DISPOSITION: 

(X)  Action  is  Considered  Complete, 

INTERNAL  MANAGEMENT  CONTROL  WEAKNESSES: 

(x)  Nonconcur 

ACTION  OFFICER:  Robert  Schmitt,  AQCOG 

PSE  RE  VIEW/ APPROVAL:  Robert  P.  Scott,  Exec  Dir,  Contract  Mgmt, 
COORDINATION;  Dave  Stumpf,  DDAI 

DLA  APPROVAL: 


Final  Report 
Reference 


Revised 
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TYPE  OF  REPORT:  Audit  PURPOSE  OF  INPUT:  Initial  Position 

AUDIT  TITLE:  DCMC  Management  of  Quality  Assurance  Resources 
(Project  No.  3CF-0071) 

FINDING  B;  DCMC  did  not  know  how  many  quality  assurance 
specialists  were  needed  to  perform  the  quality  assurance  mission 
because  DCMC  does  not  have  an  effective  method  for  determining 
quality  assurance  staffing  levels.  Specifically/  DCMC  does  not 
analyze  the  actual  work  required  to  perform  process-oriented 
quality  assurance  required  for  IQUE  and  PROCAS,  In  addition, 

DCMC  routinely  adjusts  the  previous  years’  staffing  levels 
downward  relying  primarily  on  normal  attrition  to  fit  reduced 
funding.  As  a  result,  DCMC  did  not  have  an  adequate  basis  for 
justifying  annual  funding  requests  for  quality  assurance  staffing 
required  to  perform  the  quality  assurance  mission  and  for 
determining  the  actual  imbalances  in  quality  assurance  staffing 
levels . 

DLA  COMMENTS:  Concur.  Although  DCMC  does  not  have  a  single 
command-wide  resourcing  technique,  it  should  be  noted  that  a 
variety  of  resourcing  methods  are  being  used  throughout  the 
command  (i.e.  DCMD  South  Metrics  2000,  onsite  facility  based 
resource  assessments,  and  District  resource  reviews) .  As 
indicated  in  our  response  to  Inspector  General,  DoD  Report  No. 
94-INS-12,  Inspection  of  the  Defense  Contract  Management 
Command",  September  29,  1994,  DCMC  is  developing  a  manpower 
management  program  based  upon  quantitative  and  qualitative 
workload  measurement  techniques.  The  program  objectives  include 
identifying  a  proper  resource  baseline,  and  formulating  a 
consistent  resourcing  process  to  be  used  throughout  DCMC  that 
reviews  and  projects  requirements.  The  results  and 
recommendations  of  this  report  will  be  factored  into  the  program. 

INTERNAL  MANAGEMENT  CONTROL  WEAKNESSES: 

(x)  Concur;  however  weakness  is  not  considered  material 
ACTION  OFFICER:  Robert  Schmitt,  AQCOG 

PSE  REVIEW/APPROVAL:  Robert  P.  Scott,  Exec  Dir,  Contract  Mgmt, 
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TYPE  OF  REPORT;  Audit  PURPOSE  OF  INPUT:  Initial  Position 

AUDIT  TITLE;  DCMC  Management  of  Quality  Assurance  Resources 
(Project  No.  3CF--0071) 

RECOMMENDATION  B.l:  Recommend  that  the  Director,  Defense 
Logistics  Agency,  establish  a  goal  for  developing  quality 
assurance  manpower  estimates  based  on  assessed  workload  by 
FY  1996. 

DLA  COMMENTS;  Concur.  Manpower  estimates  will  be  a  part  of 
DCMC*s  manpower  management  program. 

DISPOSITION; 

(x)  Action  is  Ongoing.  Estimated  completion  Date:  31  Dec  95 

INTERNAL  MANAGEMENT  CONTROL  WEAKNESSES; 

(x)  Concur;  however  weakness  is  not  considered  material 

ACTION  OFFICER:  Robert  Schmitt,  AQCOG 

PSE  REVIEW/APPROVAL:  Robert  P.  Scott,  Exec  Dir,  Contract  Mgmt, 
COORDINATION:  Dave  Stumpf,  DDAI 

D  D/OT, 

DIA  APPROVAL : 
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Final  Report 
Reference 


Revised 


TYPE  OF  REPORT:  Audit  PURPOSE  OF  INPUT;  Initial  Position 

AUDIT  TITLE:  DCMC  Management  of  Quality  Assurance  Resources 
(Project  No.  3CF-0071) 

RECOMMENDATION  B.2:  Recommend  that  the  Director,  Defense 
Logistics  Agency,  require  that  quality  assurance  specialists 
document  the  estimated  quality  assurance  workload  tasks  required 
for  each  contract  and  the  time  needed  to  proof  applicable 
processes,  fully  perform  mandatory  inspections,  product  audits, 
material  reviews,  investigations  of  Product  Quality  Deficiency 
Report  and  subcontract  quality  assurance.  Quality  Assurance 
specialists  should  report  the  workload  estimates  through  the 
Quality  Assurance  Management  Information  system. 

DLA  COMMENTS:  Nonconcur  with  the  specificity  of  the 
recommendation.  It  is  unrealistic  to  expect  a  finite  manpower 
estimate  for  such  activities  such  as  material  review  and 
investigations  of  PQDRs  prior  to  contract  performance.  We  do, 
however,  concur  with  the  documentation  of  estimated  manpower 
needs  at  each  facility.  The  DLAM  8000.5  process  on  Product  and 
Manufacturing  Assurance  requires  the  development  of  a  written 
plan  for  surveillance  of  the  product  assurance  and  manufacturing 
processes.  One  of  the  outputs  of  the  process  is  a  DCMC 
resource/skill  requirement.  The  method  for  maintaining 
management  visibility  will  be  addressed  by  DCMC*s  manpower 
management  program. 

DISPOSITION; 

(x)  Action  is  Ongoing.  Estimated  completion  Date:  31  Dec  95 

INTERNAL  MANAGEMENT  CONTROL  WEAKNESSES: 

(x)  Nonconcur 

ACTION  OFFICER:  Robert  Schmitt,  AQCOG 

PSE  REVIEW/APPROVAL;  Robert  P.  Scott,  Exec  Dir,  Contract  Mgmt, 
COORDINATION:  Dave  Stumpf,  DDAI 
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Final  Report 
Reference 


TYPE  OF  REPORT:  Audit  PURPOSE  OF  INPUT:  Initial  Position 

AUDIT  TITLE:  DCMC  Management  of  Quality  Assurance  Resources 
(Project  No.  3CF-0071) 

RECOMMENDATION  B.3:  Recommend  that  the  Director,  Defense 

Logistics  Agency,  require  that  quality  assurance  specialists  Revised 

supervisors  review  and  certify  the  process-oriented  quality 
assurance  actual  workload  estimates  reported  through  the  Quality 
Assurance  Management  Information  system, 

DLA  COMMENTS:  Nonconcur  with  the  specificity  of  the 
recommendation.  Concur  with  management  verification  of  workload 
estimates.  This  will  be  accomplished  during  normally  scheduled 
supervisory  visits  and  management  reviews.  Implementation  of 
DCMC*s  manpower  management  program  will  also  include  methods  for 
verifying  accuracy  of  workload  estimates. 

DISPOSITION: 

(X)  Action  is  Ongoing.  Estimated  completion  Date:  31  Dec  95 

INTERNAL  MANAGEMENT  CONTROL  WEAKNESSES: 

(x)  Nonconcur 

ACTION  OFFICER:  Robert  Schmitt,  AQCOG 
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TYPE  OF  REPORT:  Audit  PURPOSE  OF  INPUT:  Initial  Position 

AUDIT  TITLE:  DCMC  Management  of  Quality  Assurance  Resources 
(Project  No.  3CF-0071) 

RECOMMENDATION  B.4:  Recommend  that  the  Director,  Defense 
Logistics  Agency,  use  documented  process-oriented  quality 
assurance  workload  estimates  as  the  basis  for  budget  requests  for 
each  quality  assurance  resource. 

DLA  COMMENTS:  Concur.  Manpower  management  program  being 
developed  that  will  use  appropriate  factors  to  determine  quality 
assurance  workload  estimates  derived  at  the  facility  level. 

These  estimates  will  be  used  as  the  basis  for  budget  estimates  at 
each  organizational  level. 

DISPOSITION: 

(x)  Action  is  Ongoing.  Estimated  completion  Date:  31  Dec  95 

INTERNAL  MANAGEMENT  CONTROL  WEAKNESSES: 

(x)  Concur;  however  weakness  is  not  considered  material 

ACTION  OFFICER:  Robert  Schmitt,  AQCOG 

PSE  REVIEW/APPROVAL:  Robert  P.  Scott,  Exec  Dir,  Contract  Mgmt, 
COORDINATION:  Dave  Stumpf,  DDAI 

DLA  APPROVAL: 


T*A\Tr.“:;r'n  p.  r.-pp-LLi.,  rjs., 
fl.:: -Tta. 

Pi.  iJcpv-ty  Diisutor 
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TYPE  OF  REPORT;  Audit  PURPOSE  OF  INPUT:  Initial  Position 

AUDIT  TITLE:  DCMC  Management  of  Quality  Assurance  Resources 
(Project  No.  3CF-0071) 

RECOMMENDATION  B.5:  Recommend  that  the  Director,  Defense 
Logistics  Agency,  develop  standard  methodology  to  determine  the 
number  of  quality  assurance  specialists  needed  to  perforin  the 
contract  quality  assurance  mission  based  on  dociomented  estimates 
of  workload. 

DLA  COMMENTS;  Concur,  Manpower  management  program  being 
developed, 


DISPOSITION; 

(x)  Action  is  Ongoing.  Estimated  completion  Date:  31  Dec  95 

INTERNAL  MANAGEMENT  CONTROL  WEAKNESSES: 

(x)  Concur;  however  weakness  is  not  considered  material 


ACTION  OFFICER:  Robert  Schmitt,  AQCOG 

PSE  REVIEW/ APPROVAL :  Robert  P.  Scott,  Exec  Dir,  Contract  Mgmt, 
COORDINATION:  DaveStumpf,  DDAI 

DIA  APPROVAL: 
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TYPE  OF  REPORT:  Audit  PURPOSE  OF  INPUT:  Initial  Position 

AUDIT  TITLE:  DCMC  Management  of  Quality  Assurance  Resources 
(Project  No*  3CF-0071) 

RECOMMENDATION  B.6:  Recommend  that  the  Director,  Defense 
Logistics  Agency,  standardize  the  approach  and  the  rationale  for 
determining  and  describing  surplus  quality  assurance  positions 
and  for  identifying  locations  with  urgent  quality  assurance 
needs . 

DLA  COMMENTS:  Concur.  Manpower  management  program  being 
developed. 

DISPOSITION: 

(x)  Action  is  Ongoing.  Estimated  completion  Date:  31  Dec  95 

INTERNAL  MANAGEMENT  CONTROL  WEAKNESSES: 

(x)  Nonconcur 

ACTION  OFFICER:  Robert  Schmitt,  AQCOG 

PSE  REVI EW/ APPROVAL :  Robert  P,  Scott,  Exec  Dir,  Contract  Mgmt, 
COORDINATION:  Dave  Stumpf,  DDAI 

— dmt, 

DLA  APPROVAL: 
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TYPE  OF  REPORT:  Audit 


PURPOSE  OF  INPUT:  Initial  Position 


AUDIT  TITLE;  DCMC  Management  of  Quality  Assurance  Resources 
(Project  No.  3CF-0071} 

RECOMNffiNDATION  B.7:  Recommend  that  the  Director,  Defense 
Logistics  Agency,  inform  Defense  Logistics  Agency  customers  when 
Defense  Contract  Management  Command  quality  assurance  specialists 
are  not  able  to  provide  an  adequate  level  of  quality  assurance  on 
specific  source  inspected  contracts. 

DtA  COMMENTS:  Concur.  It  is  DCMC  policy  to  meet  our  customers 
expectations  and  only  the  Commander  of  DCMC  is  authorized  to  say 
no  to  a  customer  request  for  support.  The  DLAM  8  000.5  process  on 
Product  and  Manufacturing  Assurance  requires  that  appropriate 
customer  technical  representatives  be  contacted  to  establish 
points  of  contact,  clarify  requirements,  express  concerns  and 
discuss  findings. 

DISPOSITION; 

(x)  Action  is  Considered  Complete. 


Revised 


INTERNAL  MANAGEMENT  CONTROL  WEAKNESSES: 

(x)  Concur;  however  weakness  is  not  considered  material 

ACTION  OFFICER:  Robert  Schmitt,  AQCOG 

PSE  REVIEW/ APPROVAL:  Robert  P.  Scott,  Exec  Dir,  Contract  Mgmt, 
COORDINATION;  Dave  Stumpf,  DpAI 
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This  report  was  prepared  by  the  Contract  Management  Directorate,  Office 
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A .  Report  Title:  Defense  Contract  Management  Command  Management 
of  Quality  Assurance  Resources 

B.  DATE  Report  Downloaded  From  the  Internet:  01/28/99 


C.  Report’s  Point  of  Contact:  (Name,  Organization,  Address,  Office 
Symbol,  &  Ph  #):  OAIG-AUD  (ATTN:  AFTS  Audit  Suggestions) 

Inspector  General,  Department  of  Defense 
400  Army  Navy  Drive  (Room  801) 

Arlington,  VA  22202-2884 


D.  Currently  Applicable  Classification  Level:  Unclassified 

E.  Distribution  Statement  A:  Approved  for  Public  Reiease 

F.  The  foregoing  information  was  compiled  and  provided  by: 

DTIC-OCA,  Initials: _ VM _ Preparation  Date  01/28/99 


The  foregoing  information  should  exactiy  correspond  to  the  Titie,  Report  Number,  and  the  Date 
the  accompanying  report  document.  If  there  are  mismatches,  or  other  questions,  contact  the 
above  OCA  Representative  for  resolution. 
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